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Reader’s Guide to APS Processes and Procedures 


The Center for the Advancement of Engineering Education (CAEE) began in January 2003 with 
a grant from the National Science Eoundation (EST0227558). Two NSE Directorates, 
Engineering and Education and Human Resources, oversee the Center's work. The Academic 
Pathways Study (APS) is part of the Scholarship on Eearning Engineering element of the CAEE. 

This document provides a picture of APS study design and implementation activities. 
Descriptions of the APS analysis methods, findings, and the more technical aspects of the 
research such as methodological background, sampling calculations, statistical methods, etc. are 
being reported elsewhere. An up-to-date listing of papers and reports emanating from the APS 
research as well as contact information can be found at http://www.engr.washington.edu/caee/ . 

The chapters in this document progress more or less chronologically. The following descriptions 
of each chapter provide a quick overview to orient the reader to the content of this document. 

• Chapter 1 presents the overall goals and background for the APS research. This chapter 
includes information such as research and leadership team description, study design, 
participating school descriptions, data storage and analysis plans, and study terminology. 

• Chapters 2 through 5 cover the longitudinal portion of the study, one chapter for each of the 
four years of longitudinal research on a cohort of 160 engineering undergraduates. These 
chapters contain information on recruitment, study group assignments, changes to original 
study design, data collection using four primary methods, and miscellaneous notes and 
reflections. 

• Chapter 6 covers the Broader Core Sample (800-1- students from the schools participating in 
the longitudinal study) and Chapter 7 covers the Broader National Sample (4200-1- students 
from 21 institutions around the country). These two chapters describe CAEE’s use of the 
APPEE (Academic Pathways of People Eearning Engineering) survey to confirm findings 
from the longitudinal work discussed in Chapters 2 through 5. The two chapters include 
information about IRB approval, recruitment, sampling plan, data collection/survey 
deployment, and miscellaneous notes and reflections. 

• Chapter 8 covers the school-to-work transition with the Workplace Cohort, describing the 
general methodology and the three threads of workplace data collection and analysis. 

• The extensive appendices listed at the end of this document include study materials ranging 
from data access guidelines to sampling plans to data collection instruments. 

It is important to acknowledge the incredible team of APS researchers and staff, many of whom 
continued with the project for its entire six-year run, with time on either end of that period for 
ramping up and wrapping up. This document represents our collective knowledge about the APS 
research. It is offered as the foundational underpinnings of the Academic Pathways Study, and 
as a set of practices and strategies that may be of use to future researchers. 
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1 Background and General Information 

1.1 CAEE and APS Overview 

In 2003 the National Science Foundation funded the Center for the Advancement of Engineering 
Education (CAEE), dedicated to advancing the scholarship of engineering learning and teaching. 
CAEE is a collaboration of five schools: Colorado School of Mines, Howard University, 
Stanford University, University of Minnesota, and University of Washington. 

The largest component of CAEE was the Academic Pathways Study (APS), a multi-method 
study to describe how people navigate their undergraduate education to become engineers. The 
Academic Pathways Study was led by a senior researcher from Stanford University, with the 
principal co-investigator from each of the four core partner institutions serving on the leadership 
team (Eigure 1.1). 


Figure 1.1 APS Organizational Chart 
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As a working body, the APS leadership team had several major categories of responsibilities: 

• Developing policies, standards and procedures for handling the data, reporting findings 
(including publication and authorship protocols), dissemination, etc. 

• Coordinating the development of the research methods and their consistent 
implementation on the various campuses 

• Leading the data collection process, including Institutional Review Board applications 

• Monitoring the effectiveness and progress of the APS research team 

Each principal co-investigator was responsible for supervising the APS researchers at his/her 
school and championing a set of research instruments to be used across schools. In this capacity, 
each principal co-investigator oversaw the development, training, data processing and data 
analysis related to their instrument(s) for all campuses. Howard University served as champion 
for structured interviews, the UW for the ethnographic tools and engineering design tasks, 
Stanford for survey instruments, and the CSM for academic transcript information (to verify 
majors and provide data about coursework). 

Monthly conference calls and periodic face-to-face meetings facilitated the work of the APS 
leadership team. 

The full research team was drawn primarily from the four core partner institutions, and also 
included area-experts from other institutions. Although specific campuses were designated to 
lead different components of the research, the team collaborated on all aspects of the project 
including subject recruitment, instrument design and implementation, and data processing and 
analysis. Teamwork was fostered by face-to-face workshops of the entire APS research team, as 
well as smaller targeted cross-institutional meetings and conference calls. Such collaboration 
contributed to the robustness of research processes across campuses, domains and perspectives. 


1.2 Academic Pathways Study Design 

APS research was focused on the following questions: 

1. How do students’ engineering skills and knowledge develop and/or change over time? 

2. How does one’s identity as an engineer evolve? More specifically, how does student 
appreciation, confidence, and commitment for engineering change during the undergraduate 
educational experience? How do these changes impact student decisions about pursuing 
engineering after graduation? 

3. What elements of engineering education contribute to the students' skills/knowledge and 
identity? What do students find difficult and how do they deal with the difficulties they face? 

4. What skills do early career engineers need as they enter the workplace? Where did they 
obtain these skills? Are any skills missing? 

To address these research questions, the overall study design included four cohorts and a variety 

of data collection methods. The study design as originally conceived is shown in Table 1.2, with 

notes indicating later modifications to the design. Figure 1.3 shows the design for the 
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Longitudinal Cohort research. In addition to the data sources listed in the figures, key statistics 
such as SAT scores and major status were collected for the Longitudinal Cohort. 

It is important to note that certain aspects of the study design changed over the course of the 
study to maximize the use of resources and respond to conditions and lessons that surfaced along 
the way. These changes are described throughout the document, as they occurred. 

The primary goals for each cohort* were: 

• Longitudinal Cohort - Identify and characterize the pathways and decisions involved in 
becoming an engineer 

• Broader Core Sample - Validate Longitudinal Cohort findings with a broader set of 
engineering students at the same institutions 

• Broader National Sample - Validate Longitudinal Cohort findings at a broader set of 
institutions nationally 

• Workplace Cohort - Learn what goes into becoming an engineer that is not taught or 
learned as part of the academic training 


Table 1.2 Design of the Research Cohorts (Original Design with Modifications) 

Longitudinal Cohort: Students who expressed interest in majoring in engineering upon admission at 
four institutions, followed from their freshman through junior years (2003-2006). Later modified to 
extend through senior year (2007). Study group n=160 (40 per school, including 8 for ethnographic study) 


How do students’ 
engineering skills and 
knowledge develop 
and/or change? 

• Interviews* (once per year) 

• Surveys (twice per year) 

• Skill and concept-based tests and interviews (once per year) 

• Ethnographic observations of a subset of students in classes (variable) 

How do students 
develop an identity as 
an engineer? 

• Interviews* (once per year) 

• Surveys (twice per year) 

• Ethnographic observations of a subset of students in various environments 
(variable) 

What education 
challenges do students 
face? What resources 
do they draw upon? 

• Interviews* (once per year) 

• Surveys (twice per year) 

• Ethnographic observations of a subset of students in various environments 
(variable) 


Broader Core Sample: Engineering undergraduates at the four Longitudinal Cohort institutions who are 
not in the Longitudinal or Workplace Cohorts, at one point in time (2006-07); n>2000 (original design). 
Actual number of participants: 842. Survey administration in April 2007. 


Are Longitudinal Cohort 
findings representative 
of other engineering 
students at the school? 


Cross-sectional surveys developed from the evolving research results 


In some APS publications the cohorts are numbered; cohort 1 is the Longitudinal Cohort, cohort 2 is the 
Workplace Cohort, cohort 3 the Broader Core Sample, and cohort 4 is the Broader National Sample. 
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Broader National Sample: Undergraduate students from engineering programs at approximately 20 
institutions across the country, at one point in time (2006-2007); n>=3000 (original design). 

Later modified to n>1080; actual number of participants: 4266. Survey administration at 21 institutions 
during January to March, 2008. 


Are Longitudinal Cohort 
findings representative 
of other schools? 


Cross-sectional surveys developed from evolving research results 


Workplace Cohort: Students majoring in engineering at two institutions, from the end of their junior 
year through their first two years post-B.S. (2005-2007); n=16, 8 in each of two schools (original design). 
Later modified to be a cross-sectional investigation of new professional engineers employed in various 
settings; actual number of participants: 111. 


What skills do early 
career engineers’ need 
as they enter the 

workplace? 

• Ethnographic observations and comparative analyses of skills and knowledge 
used in school and at work. 

• Interviews 

How do students 
develop an identity as 
an engineer? 

• Interviews 


* To help researchers gain deeper insights, a subset of Longitudinal Cohort participants was designated to receive 
semi-structured ethnographic interviews, in lieu of the structured interviews most participants received. 
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Figure 1.3 Longitudinal Cohort research design (see Study Terminology in Section 1.9) 
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1.2.1 Participating Institutions 

Four diverse institutions provided the student base from which subjects were recruited for the 
Longitudinal Cohort and Broader Core Sample. A fifth institution provided subjects for the 
Cross-sectional Cohort which was added to the original APS design and conducted in 2005- 
2006. The five schools, identified by pseudonym, are described below. 

Use of School Pseudonyms 

Pseudonyms were adopted to describe the participating schools for use in publications and 
presentations (including this document). The rationale for using pseudonyms was to protect 
these institutions from any possible negative implications or perceptions that might come out of 
the research. Pseudonyms evolved over the course of the research with the final decision being to 
use pseudonyms that were broadly descriptive and not easily traceable to the real school names. 
School pseudonyms include Technical Public Institution (TPub), Urban Private University 
(UPri), Suburban Private University (SPri), Large Public University (LPub), and Large 
Midwestern Public University (LMPub). 

Participating Institution Pseudonyms and Descriptions 

Technical Public Institution (TPub) is a public research university devoted to engineering and 
applied science (2004 Carnegie Classification: Specialized Institution-Engineering). In 2004- 
2005, 75 percent (2,500) of its 3,350 students were enrolled in undergraduate programs, with 
approximately 600 of those being entering freshmen. Students face a rigorous curriculum and 
high academic standards. In 2002, TPub graduated a total of 539 undergraduates, 440 of whom 
received degrees in engineering majors (i.e.. Chemical Engineering, (General) Engineering, 
Geology and Geological Engineering, Geophysics, Metallurgical and Materials Engineering, 
Mining Engineering, Petroleum Engineering, and Engineering Physics). 

Urban Private University (UPri) is a comprehensive, historically Black private university (2004 
Carnegie Classification: Doctoral Research-Extensive). UPri offers an abundance of 
extracurricular associations and activities, promoting a sense of family among the student body. 
Of the 10,000 students at UPri in 2004-05, approximately 1400 were freshman, with 180 
entering the engineering program each year. Ereshmen are accepted into the engineering program 
upon enrollment. Engineering majors offered include Chemical, Civil, Systems and Computer 
Science, Mechanical, and Electrical Engineering. In 2002, UPri graduated a total of 108 students 
from engineering programs. 

Suburban Private University (SPri) is a private research university, with an enrollment of 
about 14,000 students, divided equally between graduate and undergraduate students (2004 
Carnegie Classification: Doctoral Research-Extensive). SPri attracts students from around the 
nation and the world, with fifty percent of students classified as non-Caucasian. Of the 1600 
freshmen entering each year, 320 to 350 self-identify as being interested in engineering. 

(Entering freshmen do not formally declare majors.) In 2002, a total of 373 students graduated 
from undergraduate engineering programs, including 154 students in Computer Science. 
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Large Public University (LPub) is a very large public research university (2004 Carnegie 
Classification: Doctoral Research-Extensive). Over 40,000 students attend LPub. The main 
campus offers a variety of outdoor activities in close proximity. Students tend to form 
associations and friendships based on shared academic interests. Of the 7,000 entering freshmen 
each year, approximately 650 are designated pre-engineering prior to their arrival. Admission 
into the highly competitive undergraduate engineering program typically occurs during the 
summer before the junior year and many potential applicants move into other fields before then. 
In 2002, undergraduate engineering programs at LPub graduated a total of 659 students. 

Large Midwestern Public University (LMPub) is a very large public research university (2004 
Carnegie Classification: Doctoral Research-Extensive) with over 50,000 students (2008). Seven 
engineering departments are combined with mathematics and the physical sciences under the 
umbrella of one technical college. Lirst and second year engineering students take foundation 
math and science courses in this technical college. At the end of their second year they must 
petition for admission to the upper division and a specific engineering department. In 2008, 
approximately 3,300 of 4,600 undergraduates in the technical college were engineering students. 

1.2.2 Ensuring Diversity 

Including students from diverse backgrounds was a key element of the research plan. In the 
Longitudinal Cohort, we paid special attention to understanding how underrepresented students 
navigate their initial years in engineering education. We accomplished this by employing over- 
sampling strategies for gender (male/female) and underrepresented minority* students, including 
African American/Black, American Indian/ Alaska Native, Mexican American/Chicano, Puerto 
Rican, other Latino groups. In the Broader Core and National Samples, recruitment targets for 
females and underrepresented minorities, plus strategic recruitment efforts, ensured that diversity 
considerations carried through to these cohorts also. 

1.2.3 Protecting Identities of Participants 

Longitudinal Cohort participants were assigned study ID codes that contained no personal 
identification information. These IDs consisted of the school code, the cohort code (01), a single- 
digit gender code (M or L) and a five-digit number assigned by the local research team. In 
addition, the students who participated in the ethnographic study were assigned pseudonyms for 
ease of reference among the research team. To avoid influencing how these participants may be 
treated by advisors and faculty, participants were not identified to faculty or other students, 
including those involved in the research. 

Participants in the Broader Core and National Samples submitted data anonymously. The only 
identifying information was the email address subjects provided in order to claim their incentive. 
These addresses were released only to the payments coordinator for the purpose of issuing 
incentives. 


’ We defined underrepresented minorities as those traditionally underrepresented in engineering education relative to 
their representation in the general population. See Chubin, D., May, G., and Babco, E. "Diversifying the 
Engineering Workforce" Journal of Engineering Education, Vol. 94, No. 1, 2005, pp. 73-86, and May, G., Chubin, 
D. "A retrospective on Undergraduate Engineering Success for Underrepresented Minority Students " Journal of 
Engineering Education, Vol. 83, No. 1, 2005. 
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1.2.4 Incentives to Participate 

Students in the Longitudinal Cohort received $175 per year of participation. In Year 1, they also 
received a donated scientific calculator. Students designated for the control group (described in 
section 1.9) were to receive $25 annually, but the group was disbanded in Year 2. Subjects in the 
Broader Samples were offered $4 through a popular online financial transaction company. 

1.3 Data Storage, Organization and Access 

APS employed a database consultant to oversee all aspects of data storage, organization, security 
and access. Data were stored on secure servers on one of the partner campuses. The database 
consultant participated fully with the research team to stay abreast of research activities and 
generally ensure the smooth functioning of all data-related systems. 

1.3.1 Technology Infrastructure 

APS used an online collaboration system, the APS Workspace that functioned as a secure 
database allowing team members to coordinate data collection activities and share datasets and 
analysis activities. Consistent with IRB privacy concerns and the sensitive nature of the data, 
access and sharing were facilitated and carefully controlled to maintain security. A secure, web- 
based infrastructure built on wiki technology enabled researchers to quickly view and share 
information from anywhere via the internet. Furthermore, the Workspace was organized in a 
fashion that allowed it to grow organically, making it possible to add file storage areas, blogs and 
private workspaces. 

1.3.2 Security and Backup 

Like most research, the value of the APS is inextricably tied to the data. Accordingly, extreme 
care and attention were devoted to data security and backup. On an hourly basis, data from the 
APS Workspace were backed up to a primary computer and a secondary backup computer. 
Nightly backups were made to a secure off-site storage machine. 

1.3.3 Data Collection and Inventory 

The APS utilized a detailed file naming convention that included codes for school, cohort, 
gender, individual ID, research method, and more (Appendix I-A). This file naming system 
allowed researchers to quickly identify a file's origin, purpose and status. Given the different 
needs of the different research methods, not all data were stored in a single database. As a result 
database queries were possible within a given database, but not system-wide or across databases. 

1.3.4 Access Policy 

To protect the privacy of APS participants and facilitate adherence to Institutional Research 
Review Board (IRB) procedures and obligations, access to APS research data was governed by 
criteria set forth in the APS Data Access Guidelines, included in Appendix 1-B. The goal of 
these guidelines was to minimize the likelihood for accidental data sharing with those for whom 
data access may constitute conflicts of interest or violate IRB approved research protocols and 
privacy laws. 
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Basically, researchers from each of the core partner schools had access to their school's data, and 
members of each method team had access to their method's data. Beyond that, researchers could 
request data according to the Guidelines, whereby the "owner" of the data (either the school or 
the method lead) granted access. 

1.4 Data Analysis Plan and Processes 

APS generated a number of distinct data sets corresponding to different data collection 
instruments and different cohorts. For all data sets, the first line of analysis was instrument- 
specific; only data from that instrument were used in the analysis. The champion institution (i.e., 
the one leading development and implementation of the instrument) also led the instrument- 
specific data analysis for all schools. 

A second line of analysis extended across instruments and methods, utilizing data from more 
than one APS data set. 

Access to and sharing of data was managed through the online APS Workspace with a secure 
database system. Access to APS data was carefully controlled to ensure that IRB guidelines were 
observed and data was used appropriately. 

1.5 Study Terminology 

Below is a listing of terms as they apply to the APS. 


APPLES 


Control group 


Core (partner) 
institutions 


Academic Pathways of People Learning Engineering Survey 
(APPLES), derived from the PIE Survey. This web-based survey was 
the primary data collection instrument for the Broader Core and 
National Samples. Also called the APPLE survey. 

A sub-group of the Longitudinal Cohort who would not receive 
surveys or interviews. Due to unanticipated difficulties in recruiting 
adequate numbers of study participants, the control group was 
disbanded in Year 2. Also called the comparison group. 

The four educational institutions that conceived and executed the APS 
research. 


Engineering design Short problem-oriented question administered to subjects in the 
task Longitudinal Cohort as part of the annual interview. Responses 

contributed to the LTD data set. Also called scoping task, 
performance task or engineering task. 

LTD The Engineering Thinking and Doing (LTD) component of the APS 

research, designed to uncover frameworks students bring to 
engineering problem-solving. LTD included the engineering design 
tasks and specific survey questions focused on engineering design. 
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Ethnographic methods 


Exit interview 


High Contact Group 


Eow Contact Group 


Medium Contact Group 


Participating institutions 


Persister 


PIE survey 


Non-persisters 


Structured interview 


Semi-structured ethnographic interviews, field observations, and 
informal conversations. 

An ethnographic interview administered to Eongitudinal Cohort 
participants who declared a non-engineering major. 

A sub-group of Eongitudinal Cohort participants who were studied 
using ethnographic observations and semi-structured ethnographic 
interviews, in addition to surveys. This group was also referred to as 
the Ethnography group, the Ethno 8 (8 students per school) or the 
Ethno 32 (total of 32 students in the group). 

A sub-group of Eongitudinal Cohort participants consisting of 24 
students from each school who participated only in structured 
interviews, engineering design tasks and surveys (i.e., no semi- 
structured ethnographic interviews or observations). Also called The 

24. 

The eight Eongitudinal Cohort participants at each school who 
received semi-structured ethnographic interviews but no ethnographic 
observations. Because these students received both structured and 
semi- structured ethnographic interviews in Year 1, they were 
sometimes called the combo group. 

Universities from which student participants were drawn for the 
Eongitudinal Cohort, Broader Core Sample, and Cross-sectional 
Cohort. 

A student who had entered university with intent to study engineering 
and whose declared major at the end of the study period was in the 
school of engineering at that student's institution. Engineering majors 
varied by institution (e.g. some schools placed computer science in the 
school of engineering while others did not). See Appendix 1-C for 
related definitions. 

Persistence in Engineering (PIE) survey, patterned after existing 
surveys of engineering students for web-based administration to 
Eongitudinal Cohort participants. 

Individuals who declared a non-engineering major after indicating 
intent to major in engineering. 

A series of questions designed to address specific research topics. 
Structured interviews were used with Eongitudinal Cohort participants 
who did not receive the semi-structured ethnographic interview. Also 
called formal interview. 
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Semi-structured 
ethnographic interview 


Underrepresented 
ethnic groups 


A series of questions or prompts open-ended in nature designed to 
elicit free-flowing accounts of participants’ perspectives and 
experiences. The questions were designed to enable students to reflect 
upon their past, present, and future life-world experiences related to 
engineering. Semi-structured, ethnographic interviews were used with 
the High and Medium Contact Groups. Also called informal 
interview, unstructured interview or ethnography interview. 

Ethnic groups traditionally underrepresented in undergraduate 
engineering programs in the U.S., including African American/Black, 
Latino/a, and Native American. Also called underrepresented 
minorities or URM. 
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2 


Research Year 1: Fall 2003 - Spring 2004 


2.1 Tasks and Goals 

The main tasks for Year 1 were to: 

• Recruit and enroll 320 freshmen (160 in the study group and 160 in the control group) at 
the four core institutions. Students would be followed through the end of their junior 
year. * 

• Administer surveys (winter and spring) and interviews (spring) to all Longitudinal Cohort 
participants, and conduct ethnographic observations of the High Contact Group 
throughout the academic year. 

2.2 Recruitment 
2.2.1 Methods 

Recruitment activities varied at each of the four institutions, as described below. Recruitment 
efforts were tied to school calendars, with the semester schools (Technical Public Institution and 
Urban Private University) beginning in late summer and the quarter schools (Suburban Private 
University and Large Public University) starting up in early fall. Recruitment activities continued 
throughout the first school term. 

All study participants were required to sign a consent form approved by the Institutional Review 
Board (IRB) at their institution (see Appendix 1-D). To boost recruitment, a national electronics 
company donated scientific calculators to give to study participants in Year 1, in addition to the 
$175 incentive per participant per year. 

Technical Public Institution 

Recruitment efforts began during summer 2003, at four summer campus events that constituted 
“Explore TPub”: 

• The Information Fair in August 

• Two campus events sponsored by the Minority in Engineering Program (MEP) 

• An event sponsored by Society of Women Engineers (SWE) 

• An all-campus event held in the first two weeks of school. 

Once classes had begun, a member of the TPub research team made presentations at Chemistry 
lectures attended by all first-year students. She also met with residence hall assistants. Further 
outreach was planned through fraternity and sorority houses, but this step was not needed. 

Urban Private University 
Recruitment activities included: 

• A presentation about the study in July at a summer program for incoming freshmen 
(“Pre-Freshmen Summer Experience”). Interested students submitted information forms. 


’ In September of 2005, the National Science Foundation provided supplementary funds to allow researchers to 
follow participants in the High and Medium Contact Groups for an additional year. 


12 


APS Research Processes and Procedures 

May 2010 



• Presentation and brochures at student orientation in August, where more student interest 
forms were collected. 

• Advertisement of the APS informational session in September through: 

o Flyers in the Engineering and Architecture buildings 

o Invitational emails to students who had previously expressed interest in the study 
o An announcement in the “Intro to Engineering” class, which all engineering 
students attend 

• A second recruitment drive in October consisting of: 

o A table with flyers and brochures in the Engineering Building 
o A follow-up visit to the “Intro to Engineering” class 

• Contacting students who had previously expressed interest, via phone and e-mail, through 
mid-November. 

Students who decided to participate in the study signed consent forms at the information session 
in September, or in the course of recruitment activities during October and November. Eor 
students under 18, parents were contacted to sign the consent form. 

Suburban Private University 

Potential participants were contacted using the following methods: 

• Personalized letters to students who had listed engineering as a preliminary academic 
interest (September) 

• New Student Orientation presentations (September) 

• Elyers in dorms, classrooms, libraries, engineering buildings, etc. (September) 

• Group information session (October) 

• Individual e-mails to students who had expressed interest in engineering and/or in the 
study (November) 

• Mass e-mails to campus engineering societies and freshman engineering seminars 
(October, November, and December) 

• Individual information sessions (October, November, and December) 

• Engineering society meetings (October and November) 

Consent forms were signed at group and individual information sessions. 

Earge Public University 

The initial attempt to recruit Eongitudinal Cohort participants took place from late October 
through mid-November 2003. Activities included: 

• In-class presentations for courses in the math sequence, chemistry sequence, and physics 
sequence, as well as in ENGR 100, an introductory engineering class open to freshmen. 
Interested students completed "statements of interest." 

• Notices posted to the email list-server for pre-engineering students (weekly posts during 
November). 

• Information sessions, to which interested students from the above two activities were 
personally invited. Information sessions were held several times a week throughout 
November and into early December, drawing from one to four students. 
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2.2.2 Diversity Considerations 

A recruitment objective of APS was to over-sample certain populations to gain information 
about a broad range of students. To this end, sampling goals were set forth, including: 

• Obtain a gender balance of 50/50 (equal numbers of women and men) 

• Adjust sampling to include at least 25% underrepresented ethnic populations (African 
Americans, Native Americans, and Latinos) 

• Over-sample students who exhibit a keen interest in engineering (i.e., possess indicators 
that they are very likely to succeed and be retained as engineers). The intent was to 
maximize the number of persisters in the study. 

The original sampling plan for the Longitudinal Cohort is shown in Appendix 2-A. 

Urban Private University, which is predominately African-American, drew about 20 percent of 
its freshman engineering class from international (non-U.S.) students. The sampling goal for 
underrepresented students at UPri was to obtain a 50/50 balance of U.S. and non-U.S. students. 

At TPub and LPub, initial recruitment efforts yielded fewer women than men. At both schools, 
targeting women via e-mail communications helped increase the number of female participants. 
TPub researchers used e-mail to invite female students to two additional information sessions, 
while LPub researchers increased their female participation with a list-server announcement 
targeting women. Relatively few women study engineering at SPri and this was reflected in 
SPri's recruitment numbers. 

No African-American students attended information sessions at TPub. In fact, there were only 
six African American students in the freshman class of 750, and only five of them were eligible 
for APS. Various remedial strategies for recruitment were considered, including having upper- 
class students contact these first- year students. However, it was decided not to pursue them for 
fear of making students uncomfortable. Furthermore, since there were few females at TPub from 
ethnic minority groups, ethnic-minority males were over-sampled instead. 

Figure 2.1 illustrates the gender breakdown of the 160 students who began the study, while 
Figures 2.2 and 2.3 show the ethnic make-up of the 156 students who completed the first 
survey.* Gender and ethnicity data were obtained from demographic questions on the survey; 
non-responders are not represented in the figures. Further demographic details are included in 
Appendix 1-E. 


* Ethnicity data was obtained from a multiple-choice question that was periodically included as part of the APS 
survey. Students could select multiple responses. For purposes of this document, Mexican American /Chicano, 
Puerto Rican, and Other Latino have been combined into one category, "Latino". 
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Number of students 


Figure 2,1 Gender distribution (Year 1) 
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Figure 2.2 Ethnicity distribution by school (Year 1) 
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Figure 2.3 Ethnicity distribution for all schools (Year 1) 
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2.3 Group Assignments 

Each school was responsible for randomly assigning participants to control and study groups, 
bearing in mind sampling goals with respeet to gender, ethnieity, and likelihood for persisting in 
engineering. The original research design ealled for: 

• A study group of 40 students per sehool 

o 8 students assigned to a High Contaet Group (survey, semi-struetured 
ethnographie interview and ethnographie field observation) 
o 32 students assigned to a Low Contact Group (survey, structured interview and 
engineering design task) 

• A eontrol (eomparison) group of 40 students 

Academic transcripts and declared major would be colleeted for all groups. 

Prior to beginning data eolleetion, the research team deeided to increase the number of 
participants receiving the semi- structured ethnographie interview (see seetion 2.4.1). As a result, 
8 students from the Low Contact Group were reassigned to form a new group, the Medium 
Contact Group, which did both structured and semi-structured ethnographic interviews in Year 1, 
as well as surveys and the engineering design task. 
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Of the four schools, only TPub was able to recruit enough participants in Year 1 to fully populate 
a control group. This resulted in a decision not to have a control group as part of the 
Longitudinal Cohort at any school. 

2.3.1 Methods 

Each of the four institutions developed their own procedure for assigning participants to the 
various Longitudinal study groups. 

Technical Public Institution 

Thirty Caucasian male participants were selected from a sample of 56 students. The first division 
was geographical (out of state, Big City metropolitan, other State areas). The next division was 
based on the questionnaire students had completed assessing their level of interest in the study. 
Students were eliminated if (a) their primary motivation was the cash incentive, or (b) they 
preferred a limited level of participation (i.e., monitoring of academic records only). One student 
was eliminated who had missed several recruiting sessions despite confirming by email 
beforehand. Finally, the remaining students were geographically balanced for metropolitan Big 
City area of origin and other State areas. Because one major city was heavily represented, these 
students were also balanced by major. 

Of 32 Caucasian female students, one student was eliminated whose questionnaire indicated 
preference for a limited level of participation, and one more student was eliminated who had a 
non-engineering major, resulting in a group of thirty white females for the study. 

For ethnic minority participants (male and female), the initial sample population consisted of 14 
minority males and 8 minority females. One student was eliminated because of several eligibility 
factors which could not be verified: U.S. citizenship, ethnicity, and gender. Two ethnic male 
students were eliminated because they listed “money” as the primary motivation for participating 
in the study. Students who did not choose to be a participant in the “whole show” were also 
eliminated. 

Urban Private University 

The dataset of 62 student names was sorted by 1) participation in Pre-Freshmen Program, 2) 
citizenship, and 3) gender. The objective was to have 31 students in the detailed study group and 
31 students in the control group. Each group was to contain three Pre-Freshmen participants (two 
males and one female) and 28 non Pre-Freshmen participants (ten male and seven female U.S. 
citizens, eight male and three female non-U. S. citizens). 

A systematic method was used to assign students to the study group and control group. Working 
from a numbered list, students with odd numbers by their names were placed in the control group 
and the even numbered students were placed in the study group, resulting in 3 1 study participants 
and 31 control participants. 

The eight participants for the High Contact Group were systematically selected from the study 
group. The objective was to obtain two Pre-Freshmen participants (one female, one male), two 
U.S. citizen participants (one female, one male), and four non-U. S. citizen participants (two 
female, two male). The only female Pre-Freshmen participant and first male were selected from 
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the Pre-Freshmen participants. For U.S. citizen participants, the first female and the fourth male 
were selected. For non-U. S. citizen participants, the first and third female and the second and 
sixth male were selected. 

With equal participants in the detailed study and control group, it was recommended that nine 
students in the control group be added to the study group, to bring the total to 40. A systematic 
method was employed. From the Pre-Freshman Program, the first male was selected (no female 
students remained to be selected). From the U.S. citizen pool, every third female was selected, 
for a total of two females, and every fifth male was selected, for a total of two males. From the 
non-U. S. citizen pool, the first and third female was selected, for a total of two females, and 
every fourth male was selected for a total of two males. 

Suburban Private University 

A total of 44 students signed consent forms, including one student who declined to participate in 
the study. Forty students were selected to participate in the study group, and the remaining 3 
were assigned to the control group. 

The 3 students assigned to the control group were not selected randomly. One student declined to 
participate in any of the research methods, with the exception of the collection of the student’s 
academic information. Two students had not responded to updating their consent forms. 

Participants were selected for the High Contact Group based on the following criteria: 

• Gender representation: 4 females, 4 males (desired 4/4) 

• Ethnic representation: 2 ethnic minority students (desired 2-4) 

• Engineering Bridge Program: 1 student who participated in the SPri Summer Science 
Engineering Academy program during the summer before their freshman year (desired 1- 
2) 

• Academic intent: Which majors has this student indicated an interest in based on 
individual meetings with research team and preliminary academic interest information: 

o Engineering only: 6 (desired 5-6) 
o Engineering or Computer Science: 1 (desired 0-1) 
o Engineering or Physical Sciences: 1 (desired 0-1) 
o Engineering or Social Science/Humanities: 0 (desired 0-1) 

Additional qualities of the group were considered to help decide between multiple students who 
clustered together after the initial criteria were considered: 

• Extracurricular activity: students involved in a time-intensive extracurricular activity 
such as athletics: 2 selected (1 walk-on athlete, 1 varsity athlete) 

• Residential assignment: students from a freshman dorm or other special residential 
considerations: 2 selected (1 freshman/sophomore dorm, 1 whose roommate is also in the 
study) 

• Engineers in family (students whose parent is an engineer): 1 selected 

• Relationship with ethnographer: all selected students had some connection to 
ethnographer 

• High School academic information: student who had lower Math SAT scores (below 700) 
than others in the group 
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Large Public University 

Initially, students had been randomly assigned to the study group and to the control group. 
However, because of drop-outs and lack of response from some students in the study group, all 
students who expressed interest in participating in the study were eventually selected for the 
study group. 

During information meetings, students were asked to indicate in which groups they would be 
interested in participating. Five women and 15 men expressed an interest in being considered for 
the High Contact Group. 

Of the five women, two indicated they are members of underrepresented minority groups, and 
were immediately invited to participate in the High Contact Group. Two of the remaining three 
women were randomly selected for invitations to this group. One accepted and one declined so 
was reassigned to the Low Contact Group. The remaining woman also declined. In mid-January, 
a woman who had been assigned to the study group was asked if she would consider switching to 
the High Contact Group, on the basis of her enthusiasm for the study. She accepted, resulting in 
four women being assigned to the High Contact Group. 

Of the 15 men, two Caucasian males were selected for the High Contact Group on the basis of 
the researchers’ judgment that they would make good ethnographic informants. A third, an 
Asian- American, was chosen because he was the only Asian male to volunteer for the High 
Contact Group. The fourth, a direct admit to the Electrical Engineering department, was invited 
because it was expected that the experiences of directly admitted students would differ from 
those of students competing for admission to departments. 

2.3.2 Numbers 

Table 2.4 shows gender and ethnic breakdowns by school and study group. 
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Table 2.4 Gender and Ethnic Breakdowns (Year 1) 

Citizenship and Race/Ethnicity data were obtained directly from subjects' responses on Survey 1 (Winter 2004). 
Total N=156 (4 of the original subjects did not take Survey 1 and are therefore not represented in these tables) 



Gender 

Citizenship 

Race/Ethnicity 



Female 

Male 

US 

Citizen 

Non- 

Citizen 

Afr-Am/ 

Black 

Native 

Amer 

Asian 

Amer 

Latino* 

Cauc 

Other 

Multi 

URM** 

TPub 

19 

21 

40 

0 

0 

0 

4 

2 

28 

0 

6 

2 

5% 

UPri 

13 

23 

21 

15 

30 

0 

0 

0 

0 

4 

2 

30 

83% 

SPri 

12 

28 

34 

6 

5 

0 

8 

1 

18 

4 

4 

6 

15% 

LPub 

17 

23 

35 

5 

0 

0 

15 

2 

20 

0 

3 

2 

5% 

Total 

61 

39% 

95 

61% 

130 

83% 

26 

17% 

35 

22% 

0 

0% 

27 

17% 

5 

3% 

66 

42% 

8 

5% 

15 

10% 

40 

26% 



Gender 

Citizenship 

Race/Ethnicity 



Female 

Male 

US 

Citizen 

Non- 

Citizen 

Afr-Am/ 

Black 

Native 

Amer 

Asian 

Amer 

Latino* 

Cauc 

Other 

Multi 

URM** 

High 

Contact 

16 

16 

26 

6 

8 

0 

5 

3 

14 

0 

2 

11 

34% 

Medium 

Contact 

16 

16 

25 

7 

9 

0 

5 

1 

11 

2 

4 

10 

31% 

Low 

Contact 

29 

63 

79 

13 

18 

0 

17 

1 

41 

6 

9 

19 

21% 

Total 

61 

95 

130 

26 

35 

0 

27 

5 

66 

8 

15 

40 

26% 


* Latino combines Mexican, Puerto Rican and other Latino 

** URM = underrepresented minority groups including African American/Black, Native American, and Latino 
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2.3.3 Replenishing Study Groups 

Each school developed a plan for replenishing study groups in case of attrition in future years of 
the study. 

TPub planned to replace students in the study groups, as needed, with students from the control 
group. Because of the time-intensive nature of recruiting and uncertain value of maintaining a 
control set, students in the control group would not be replaced. During Year 1, TPub replaced 
two male students who left the study with two male students from the control group. One of 
these was selected because he stood out as being especially talkative during the recruitment 
process. The other replacement student had expressed specific interest in being in the study 
when he was assigned to the control group. 

UPri planned to post flyers advertising the study in the College of Engineering, Architecture, and 
Computer Sciences. Brochures would be distributed to students and faculty, and members of the 
research team would make presentations in sophomore level physics and engineering courses. 
Also, the study would be publicized during the weekly meetings of student engineering societies, 
such as the National Society of Black Engineers, the Society of Women Engineers, American 
Society of Mechanical Engineers, and American Society of Civil Engineers. 

SPri planned to recruit subjects for the control group who in turn could replace subjects in the 
study group if needed. There were concerns about the demographics of the study group and 
whether there would be at least 30 study participants with declared engineering majors at the end 
of year 2. The recruitment plan included 1) contacting professors of targeted courses and 
introductory seminars, 2) posting flyers in key building areas on campus, 3) emailing of 
announcements to engineering societies and sophomore dorm mailing lists, 4) announcements 
during class lectures and society meetings, and individual meetings with interested students. 

EPub planned to replenish subjects in the High Contact Group with ones from the Medium 
Contact Group, and subjects in the Medium Contact Group with ones from the Low Contact 
Group. If further replacements were needed, they would be drawn from the control group. The 
primary consideration in selecting students would be their willingness to participate and a sense 
that the student would be a good informant. Within this plan, attempts would be made to replace 
withdrawing subjects with demographically similar subjects. Additional recruitment was planned 
if more subjects were needed. 

2.4 Changes to Study Design 

During the recruitment phase of the study, several unanticipated developments caused the 
research team to reexamine and ultimately modify certain aspects of the original study design. 
These changes included: 

• Designating a Medium Contact Group 

• Ending recruitment efforts before the full 80 students per school had been recruited 
The rationale and implications for these changes are discussed below. 
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2.4.1 Medium Contact Group 

Researchers became concerned about losing participants from the High Contact Group, which 
would diminish this very rich source of data. In order to offset any such losses, eight students 
from the Low Contact Group at each school were designated to constitute a new group of 
participants - the Medium Contact Group - which would receive both structured and semi- 
structured ethnographic interviews. These students would not receive ethnographic observations 
except in cases where a student from the High Contact Group migrated out of the study and was 
replaced by a student from the Medium Contact Group. 

2.4.2 Recruitment Goals 

All schools except TPub experienced difficulties in identifying and recruiting adequate numbers 
of APS eligible students. Recruitment efforts were extended into November 2003 in hopes of 
increasing participant numbers. This was successful to a point. Each school was able to recruit at 
least 40 students — enough to comprise a study group. Rather than prolong the recruitment period 
further, the research team decided to cease recruitment activities at this point and commence 
with the data collection phase of the study. 

2.5 Data collection 
2.5.1 Methods 

The APS used multiple methods to collect data on participants. The main methods included 
survey, structured and semi-structured ethnographic interviews, engineering design tasks and 
ethnographic field observations. Surveys, structured interviews, and engineering tasks provided 
data on a large set of participants, while ethnographic methods (e.g., semi-structured 
ethnographic interviews and field observations) yielded deeper, richer information on a more 
limited number of students. In addition, academic transcripts were collected for all subjects. Each 
method provided a set of insights that informed the other methods and allowed emerging 
findings to be explored with the broader study population. Interpreted and analyzed together, 
data from the various methods resulted in rich descriptions of students' academic pathways, as 
well as the critical factors, challenges, and strategies related to navigating these pathways. 

The APS survey - also called the Persistence in Engineering (PIE) survey - was used to 
identify and characterize the fundamental factors that influenced students’ intentions to pursue an 
undergraduate engineering degree and, upon graduation, practice engineering as a profession. It 
covered a broad range of issues including students' attitudes about engineering, confidence in 
their abilities, aspirations, perceptions about the engineering education climate, and perceptions 
of their behaviors and experiences inside and outside the classroom. 

Survey design began with development of conceptual constructs and survey questions generated 
from a review of engineering education literature and previous national surveys of undergraduate 
education. The development team piloted the survey and conducted internal consistency analyses 
to validate survey constructs. In the course of the APS longitudinal research, survey items and 
constructs were iteratively evaluated and refined as successive survey administrations revealed 
new information. 
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There were 26 specific survey constructs, which included persistence in engineering, motivation, 
satisfaction with collegiate experience, curriculum overload, and more. PIE survey constructs are 
delineated in Appendix 4-B. 

The web-based PIE survey was administered to all 160 Eongitudinal Cohort participants twice 
per academic year for Years 1 through 3 of the study, and once during Year 4. Based on findings 
from the Eongitudinal Cohort, the survey was refined and shortened for administration to the 
Broader Core and National Samples. Copies of the PIE surveys are included in Appendix 4- A. 

Structured interviews had a set format with pre-defined questions, allowing for collection of 
specific information related to engineering education and identity and skills development. 
Interviewers could prompt participants to expand on their answers, thereby adding depth and 
texture to individual responses. While interviewers controlled the content of the interview, 
students were able to provide as much information and detail as they needed to tell their stories. 
Structured interviews were administered once per academic year to the Eow Contact participants; 
interviews were recorded and transcribed. The interview was approximately one hour in length. 
Copies of all structured interview protocols are included in Appendix 3-A. 

Engineering Design Tasks provided data specific to skill development. These tasks were 
problem-oriented activities administered in written form. The Year 1 task consisted of a free- 
response question asking respondents what factors they would consider in approaching a specific 
engineering design problem. Then an interviewer asked respondents a number of follow-up 
reflection questions. Data gathered from this activity were used to assess how broadly students 
perceived basic engineering problems, and how this changed over time. An engineering design 
task was administered annually at the end of the structured interview.* Copies of all engineering 
design tasks and administration protocols are included in Appendix 3-B, along with a table of 
other data sources that contribute to Engineering Thinking and Doing (ETD). 

Ethnographic methods allowed for collection of rich, in-depth descriptions of the culture and 
experience of engineering education through the eyes of students. By capturing individual 
student narratives, researchers were better able to discover and describe student perceptions and 
motivations, and how these contributed to educational decisions and pathways. 

Ethnographic methods helped APS researchers answer questions relating to identity development 
in undergraduate engineering majors, including the role engineering education plays. Identity has 
been cited as a key factor in retention of students in the discipline. 

• Ethnographic field observation of participants occurred during activities that were 
significant to their educational experiences such as: project work in lab-type engineering 
courses; examination periods; senior design/capstone projects; and extra-curricular 
activities. To get a sense of the day-to-day experiences of students, researchers conducted 
“day in the life” observations of students. Observations were conducted by trained APS 
researchers and recorded principally as field notes for subsequent analysis. Originally, 
each High Contact participant was to be observed for approximately 30 hours per 


’ After Year 1, an engineering design task was administered to all study participants after either the structured or 
semi-structured interview. 
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academic year - a goal that was later modified due to the time intensive nature of the 
activity (see section 2.6.4). 

• Semi-structured (ethnographic) interviews used interviewing methods that enabled 
researchers to glean aspects of culture and everyday life experiences through open-ended 
questions. APS researchers developed ethnographic interviews to reflect engineering 
student perceptions about past, present and future experiences. This approach allowed 
students to describe the culture of engineering education through their own eyes and 
impart what meaning it had for them. The annual semi-structured ethnographic interview 
was approximately 2 hours in length. Interview guides for conducting semi-structured 
interviews are included in Appendix 3-D. 

• Informal conversations were conducted throughout the study, to varying degrees at each 
of the four schools. Informal conversations allowed researchers to check in with students 
as needed to stay abreast of any changes in student status. These conversations added to 
the understanding and description of individual participants. 

Academic Transcripts were collected for all subjects from their respective institutions. 
Academic transcripts were the final determinant of major(s) and persistence in engineering. 
Transcripts also provided information about coursework, GPA and date of graduation. 

2.5.2 Interview Protocols and Training 

During the fall and winter (2003-2004), the research team compiled a manual and detailed 
protocols and guides for conducting interviews (Appendix 3-C), tailored to the institution and 
type of interview (structured vs. semi-structured/ethnographic). In addition, members of the 
research team who would be serving as interviewers underwent a group training in February 
2004 and several practice sessions before conducting interviews with study participants. 

All interviews (structured and semi- structured) were recorded, uploaded to the database, and 
transcribed. 

2.5.3 Ethnography Observation Training 

In summer (2003) APS researchers participated in ethnography observation training. The two- 
day training was conducted by CAFE researchers, and included practice observations and write- 
ups that were reviewed as a group. 

2.5.4 Summary of Data Collected 

Table 2.5 summarizes Year 1 data collection activities, including numbers of participants 
engaged in each method. Academic transcripts were collected but are not included in the table. 
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Table 2.5 Data collection summary by school for Year 1 (2003-2004) 



Surveys (n=40) 

Structured Interviews (n=32) 

Semi-structured Ethnographic 
Interviews (n=16) 

Ethno Obs 
(n=8) 


Dates 

# 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'y 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'y 

# 

Hours 

spent 

TPub 

1/20- 

2/19/04 

40 











4/1- 

5/3/04 

40 

3/11- 

4/22/04 

32 

19-51 

32 

3/11- 

4/6/04 

16 

n/a 

0 

34 

34 

UPri 

1/20- 

3/8/04 

36 











4/7- 
6/1 6/04 

36 

3/23- 

5/4/04 

28 

20-55 

28 

3/23- 

5/11/04 

16 

45-105 

0 

15 

30 

SPri 

1/20- 

2/9/04 

40 











5/10- 

5/23/04 

40 

4/9- 

4/30/04 

32 

18-60 

32 

4/17- 

5/12/04 

16 

54-165 

0 

30 

34.5 

LPub 

Winter 

'04 

40 





Dec '03 

2 

(pilot) 





Spring 

'04 

40 

Apr-May 

'04 

32 

n/a 

32 

Apr-May 

'04 

16 

n/a 

0 

55 

85-1- 


2.6 Notes and Reflections on Year 1 
2.6.1 Recruitment Challenges 

Recruiting students into the study proved more difficult and time-consuming than expected. The 
specific challenges varied by school, as described in reports submitted by each school's primary 
recruiter. 

Technical Public Institution 

The principal challenge at TPub was recruitment of African-American students. The goal was to 
have at least one female and one male High Contact participant who was African American. 
However, researchers at TPub were unable to recruit any African Americans into the study. 

A secondary challenge at TPub was recruiting adequate numbers of women into the study. 
Although exactly forty women signed consent forms to proceed with the study, two were 
eliminated because of ineligible majors or lukewarm interest. Furthermore, there were 
insufficient ethnic minority females to meet sampling goals. As a result, the subject pool at TPub 
included more minority men and fewer women overall. 

Urban Private University 

The main challenge at UPri was recruiting a relatively large number (80 students to fill both 
study and control groups) from a small incoming class (107 engineering students). It was 
difficult to generate interest among this small student pool. 

It was also difficult getting students to come in to sign the consent forms. Of the 85 students who 
initially expressed interest in participating, only 62 signed consent forms. 
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Suburban Private University 

Students were more available (responsive) during the very beginning of the quarter and at the 
end of the quarter, just before leaving for break. In between, it was difficult to reach freshmen 
because attendance at extracurricular activities and response to mass emails diminished greatly. 

Arranging individual meetings with the students was time-consuming but paid off in the long- 
run. The individual meetings allowed the students to ask questions and establish a connection 
with the research team. 

Large Public University 

Surprisingly few students expressed an interest relative to the number of freshman engineering 
and pre-engineering students. It was very difficult to arrange meetings with students, both for the 
information sessions and for signing informed consent. What was expected to be a month-long 
recruitment process took nearly 3 months. 

2.6.2 Consistency of Methods and Procedures across Schools 

Because of the inherent differences between the four core schools, study methods and procedures 
were not completely consistent across schools. Some of these institutional differences included: 

• Academic calendars. TPub and UPri operated on the semester system, while SPri and 
LPub used the quarter system. This meant the timing of study activities was slightly 
different at each of the four schools. 

• Freshman orientation activities. Recruitment plans for this study took advantage of 
freshman orientation activities to publicize the study and recruit students. However, 
differences in orientation schedules and programs among the schools meant that 
recruitment methods and timelines had to be customized per school. 

• Entry into engineering. The identification of engineering students for the study was 
complicated by the fact that the four schools followed different timelines and procedures 
for declaring majors. TPub and UPri students could declare an engineering major at any 
point during their freshman and sophomore years. SPri students typically did not declare 
a major during the freshman year, but rather named a broadly defined area of interest. At 
LPub, students must apply and be accepted into the engineering program (typically 
following the sophomore year) and many students are turned away. Recruitment and 
eligibility criteria were adjusted to accommodate these differences. 

• Diversity. The participating schools differed in terms of ethnic and gender mix among 
their students, making diversity goals difficult to achieve across all schools. UPri students 
were predominately African-American with a high percentage of international students. 
Therefore, UPri introduced diversity into their study population by over-sampling 
students who were not U.S. citizens. TPub had less ethnic diversity compared to the other 
schools, while SPri had relatively fewer female students in engineering. 

2.6.3 Non-Random Assignment to Study Groups 

Random assignment of participants into study and control groups as defined in the original 
sampling plan (Appendix 2- A) was possible only at TPub and UPri, where sufficient numbers 
had been recruited. At SPri and LPub, virtually everyone who signed a consent form was 
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assigned to the study group in order to reach the goal of 40 study (non-control) participants per 
school. 

Since the goal of the Longitudinal Cohort was to describe a range of academic pathways for 
engineers, students were not randomly assigned to the high, medium or low contact groups. 
Instead, group assignments were based on a variety of factors, of which ethnicity and gender 
were only two. 

Students who were unique in some way, or had a proclivity for sharing their stories, were 
considered desirable candidates for the high and medium contact groups. This approach was 
intended to yield a wide and rich array of personal stories. 

Another consideration when making group assignments was whether the student was likely to 
stay in engineering for the duration of the study. In order to achieve study goals, it was important 
to have a large majority of participants graduate with an engineering degree. On the other hand, 
it was desirable to also include the stories of students who left the field; who were they and what 
factors drove their decision to leave? 

2.6.4 Insufficient Ethnography Resources 

By the end of Year 1, it became apparent that resources for scheduling, conducting and recording 
field observations were limited at most campuses. Maintaining contact and scheduling 
observations with students in the High Contact Group was more difficult and time-consuming 
than anticipated. In addition, each hour of observation in the field required two to three more 
hours of writing up field notes. As a result, virtually no field observations were conducted in 
spring 2004 when researchers were conducting interviews. 

2.6.5 Recording and Transcribing Semi-structured Ethnographic Interviews 
Following the first year of data collection, the ethnography group at LPub hired a transcriber to 
transcribe the semi-structured ethnographic interviews from all four partner schools. Over the 
four years of data collection, the research team used a variety of individual transcribers and 
businesses to process the raw files. 
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3 


Research Year 2: Fall 2004 - Spring 2005 


3.1 Tasks and Goals 

The main tasks for Year 2 were to: 

• Manage the Longitudinal Cohort study pool to maintain size and diversity of the study 
groups as much as possible 

• Administer surveys (fall and spring) and interviews (spring) to all participants in the 
study group 

• Conduct exit interviews with students who declared majors other than engineering 

3.2 Changes to Study Design and Procedures 
3.2.1 Control Group 

Because of the unforeseen difficulty of recruiting adequate numbers of students for the study, the 
research team decided not to further pursue or maintain a control group whose only purpose was 
to provide demographic comparison with the study subjects. Existing control subjects were 
tapped to offset attrition of the study groups, as described in Section 3.3. The remaining controls 
were disbanded in the fall of 2004, as described here: 

Technical Public Institution 

After replenishing the study group, students remaining in the control group were sent a letter 
informing them they would no longer be followed as part of the study. The text of the letter 
appears below. 

Dear Academic Pathways Participant: 

The Control Group for the Academic Pathways Study will no longer be 
followed after November 3, 2004, the expiration date for our current signed 
consent. Thank you for your participation in our research. We hope to keep your 
name on file; if our research plan changes we will invite you to participate again. 

If you wish to be removed from our mailing list, please reply to this email. 

Thank you again for your participation. 


Urban Private University 

Only 12 of the 22 control subjects returned to sign consent forms in the fall of 2004. All 12 of 
these students were placed into study group slots vacated by departing study group subjects. 

Suburban Private University 

In anticipation of attrition among study group participants, the three control group subjects were 
invited to join the study group. One student agreed, increasing the study population at SPri to 41. 
Of the other two control students, one declined to participate further and the other did not 
respond, ending their participation. 
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Large Public University 

There were no control subjects to disband at LPub. 

3.2.2 Types of Data Collected 

The research team decided that in Year 2 and beyond, structured interviews would not be 
administered to subjects in the Medium Contact Group. (This group had received both semi- 
structured ethnographic and structured interviews in Year 1.) Since there was considerable 
overlap between the two interviews, this move was intended to reduce undue burden on 
participants and interview staff. 

In Year 2 and beyond, the engineering design tasks were administered to all participants, 
including those in the High Contact Group. (Students in this group did not do an engineering 
design task in Year 1.) Data gathered from engineering tasks would help researchers analyze 
how students’ perceptions of engineering and approaches to engineering problems developed as 
they progressed through the college years. In Year 2, the engineering task was administered 
immediately following the spring interview. 

3.2.3 Transcription of Structured Interviews 

Beginning in Year 2, the research team at Howard University took over the responsibility for 
transcribing structured interviews, except for the engineering task, which would be transcribed 
by a consultant. 

3.3 Migration between Study Groups 

There were three main reasons for migrations of study participants between groups: 

• Departures from the study caused by students declining to complete surveys and 
interviews, or leaving school. 

• Reallocation of students among study groups as a result of departures. 

• Switching to non-engineering majors, which triggered an exit interview. 

As can be expected, the APS study pool experienced some attrition during Year 1. A total of 
eighteen students left the study after Year 1, four of whom had never provided data. Since the 
number of subjects was not large to begin with, researchers drew from their control group 
subjects to replenish study groups during Year 2. These and other migrations are described in 
section 3.3.1. 
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3.3.1 Year 2 Migration and Replenishment 


Technical Public Institution 

Five students left TPub and the study.* One was in the Medium Contact Group, and the four 
others were in the Low Contact Group. Five students from the control group were reallocated to 
fill the vacant slots, maintaining gender and ethnic diversity as much as possible. 

Urban Private University 

Eight students left the study before the 2004-2005 academic year, in addition to four who failed 
to take even the first survey. Of the eight who left, one student transferred to another school, two 
declined to participate further, and five declared non-engineering majors. This created a total of 
twelve open slots that were filled with students from the control group. 

Suburban Private University 

All 40 study participants returned for Year 2 in their original groups. In addition, one student 
from the Control Group was added to the Low Contact Group, bringing the total number of 
participants in that group to 25. 

Large Public University 

One student (Asian- American female) switched to a business major and left the university. Her 
slot was not refilled in Year 2. 

3.3.2 Exit Interviews 

Exit interviews were administered to participants who declared a major in a field other than 
engineering. There was institutional variation in which majors were considered engineering. 

Data from exit interviews contributed to understanding why and how engineering was not 
meeting student needs, or conversely, why and how other majors may be better at meeting those 
needs. The exit interview guide is included in Appendix 3-G. 

During Year 2, Urban Private University conducted 5 exit interviews. UPri participants who 
exited the field of engineering in Year 2 were dropped from the study. (This procedure changed 
for Year 3, as described in section 3.5.1.) 

By the end of Year 2, 14 SPri participants had declared non-engineering majors and received exit 
interviews. All of these students were retained in the study in their original study groups. 

3.3.3 Study Group Demographics 

Ligures 3.1, 3.2 and 3.3 show the gender and ethnicity distributions of study participants in Year 
2. Lor subjects who were added to the study in Year 2, ethnicity was determined from an item on 
the spring 2005 survey. Subjects continuing from Year 1 retained the ethnicity identification they 
had indicated the previous year. Additional demographic details are included in Appendix 1-E. 


’ Because the Technical Public Institution is focused exclusively on engineering and applied science, study 
participants who wished to pursue other majors had to transfer to another institution 
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Number of students 


Figure 3.1 Gender distribution (Year 2) 



Technical Public Urban Private University Suburban Private Large Public University Totals 

Institution University 


Figure 3.2 Ethnicity distribution by school (Year 2) 
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Figure 3.3 Ethnicity distribution for all schools (Year 2) 


Multi-ethnic 



7 % ( 1 1 ) 


3.4 Data Collection 


3.4.1 Methods 


The methods for data eollection in Year 2 followed similar protoeols as were used in Year 1. 
Informed consent was obtained from each participant before any surveys, interviews, or 
observations were conducted. Copies of representative consent forms are included in Appendix 


1-D. 


Minor revisions were made to the Year 2 interview protocols and guides based upon questions 
emerging from Year 1 findings. These changes were reflected in an updated interviewer manual 
(Appendix 3-C). Similarly, the survey was refined to improve internal consistency (Cronbach's 
Alpha scores) of the variables. 

3.4.2 Summary of Data Collected 

Table 3.4 summarizes what data was collected by each school during Year 2. Academic 
transcripts were collected for Year 2 but are not reported in this table. 
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Table 3.4 Data collection summary by school for Year 2 (2004-2005) 



Surveys 

(n=40) 

Structured Interviews (n=24) 

Semi-structured Ethnographic 
Interviews (n=16) 

Ethno Obs 
(n=8) 

Exit 

Interv 


Dates 

# 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'v 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'v 

# 

Hours 

spent 

# 

TPub 

Fall 

2004 

40 











0 

Spring 

2005 

40 

n/a 

24 

27-59 

24 

n/a 

16 

71-179 

16 

25 

25 

0 

UPri 

Fall 

2004 

39 











5 

Spring 

2005 

38 

3/10- 

5/12/05 

23 

30-60 

23 

3/9- 

5/5/05 

14 

73-102 

14 

0 

0 

0 

SPri 

Fall 

2004 

41 












Spring 

2005 

41 

4/18- 

5/13/05 

25 

31-74 

25 

4/18- 

6/8/05 

16 

88-163 

16 

16 

22 

11 

LPub 

Fall 

2004 

39 











0 

Spring 

2005 

39 

4/18- 

5/20/05 

23 

40-120 

23 

Apr-May 

'05 

14 

90-120 

14 

40 

120 

0 


3.5 Notes and Reflections on Year 2 

3.5.1 Non-engineering Majors 

In Year 2, five students from Urban Private University switched from engineering majors at the 
beginning of the academic year and were dropped from the Longitudinal Cohort. The one student 
from LPub who switched majors mid-year was also dropped. 

To better understand or characterize students who declared non-engineering majors, researchers 
sought to retain participants who exited engineering during Years 3 and 4. 

• Students in the Low Contact Group would receive an exit interview and surveys. 

• Students in the Medium and High Contact Groups would receive exit interviews, surveys, 
semi- structured ethnographic interviews and engineering design tasks. 

The above procedures were applied to the 14 SPri students who declared non-engineering majors 
in spring 2005. 

3.5.2 Non-enrolled Students 

Two SPri participants were on leave for one academic term for personal reasons. However, this 
did not impact data collection. 

3.5.3 Missing Data 

Failing to complete a survey or interview did not automatically eliminate a participant from the 
study. Students who studied abroad fell into this category, as did others. Following a single 
participant for the duration of the study period was deemed valuable even if one or more data 
points were missing. Students who missed surveys or interviews because of disinterest or other 
factors were dropped at the discretion of each school’s research team, on a case-by-case basis. 
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4 


Research Year 3: Fall 2005-Spring 2006 


4.1 Year 3 Tasks and Goals 

The main tasks for Year 3 were to: 

• Manage study pool to maintain size and diversity of the study groups as much as possible 

• Administer surveys (fall and spring) and interviews (spring) to all participants in the 
study group 

• Conduct exit interviews with students who declared majors other than engineering 

• Plan for the Workplace Cohort and the Broader Core and National Samples 

• Begin analyzing data 

4.2 Changes to Study Procedures 

4.2.1 Non-engineering Majors 

Study participants who declared majors other than engineering were interviewed about their 
decision in an exit interview (Appendix 3-G). These students were not dropped from the study, 
as had been the case during part of Year 2. Instead, they continued in their assigned groups and 
participated in surveys and interviews, although many chose not to. 

4.2.2 Transcription of Semi-structured Ethnographic Interviews 

By the beginning of Year 3, it became apparent that the transcription of semi- structured 
ethnographic interviews was lagging. To help deal with this backlog, transcription 
responsibilities were distributed among the CAEE research groups. 

4.2.3 Extended Data Collection and Analysis 

In addition to supplemental NSE funds awarded in September 2005, another NSE supplement 
was awarded in September 2006. These two awards, totaling about $2,000,000, allowed 
Eongitudinal Cohort data collection and analyses to extend to Year 4. The Eongitudinal Cohort 
would be followed during the 2006-2007 academic year with one survey for all participants 
(spring 2007), plus semi-structured ethnographic interviews and engineering design tasks for the 
High and Medium Contact Groups, also in spring 2007. 

4.3 Migration between Study Groups 
4.3.1 Year 3 Migration and Replenishment 

Technical Public Institution 

Two students left the study during Year 3: 

• One Caucasian student from the High Contact Group transferred to a different 
institution to study engineering in a more diverse environment. A non-Caucasian 
student from the Eow Contact Group was moved into the vacated slot, creating a 
vacant slot in the Eow Contact Group that was not filled. 

• One student from the Eow Contact Group left TPub. This student was not replaced. 
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One student from the Medium Contact Group transferred to another institution after Year 3 to 
pursue a teaching degree. This student remained in the study for all of Year 3, and completed an 
exit interview at the end of the year. 

Urban Private University 

A total of six students left the study during Year 3: 

• Two students did not enroll at UPri: one from the Low Contact Group and one from the 
High Contact Group 

• Three students, all from the Low Contact Group, did not respond to survey/interview 
requests 

• One student from the High Contact Group declined to participate because she was 
studying abroad 

Vacancies created by these departures were not filled. 

Suburban Private University 

One student from the Low Contact Group withdrew from the study and was not replaced. 

Large Public University 

A total of seven students left the study during Year 3: 

• Six students were either not accepted into engineering or switched out of the program 

• One student was lost to follow-up (failed to respond) 

Of the seven students who left the study, three were from the High Contact Group and four were 
from the Low Contact Group. Three students from the Medium Contact Group were moved to 
the High Contact Group to fill the vacancies there. No other replenishment occurred. 

4.3.2 Exit Interviews 
Technical Public Institution 

Exit interviews were difficult to obtain because students who do not declare an engineering 
major typically leave TPub. Of the five participants who left TPub between Year 2 and Year 3, 
only two were reachable for exit interviews, and those were conducted in January and March of 
2006. 

One participant informed researchers of her intent to transfer after Year 3, and completed her exit 
interview prior to leaving. 

Urban Private University 

Exit interviews were conducted for three students who declared majors other than engineering. 
Two interviews were done in December 2005, and one in April 2006. 

Suburban Private University 

Exit interviews were completed for three students in March 2006. One study participant who had 
declared a non- engineering major in Year 2 returned to engineering in Year 3. 
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Large Public University 

Six exit interviews were completed in October 2005, for students who had declared non- 
engineering majors, including some who had applied but not been accepted to an engineering 
major. 

4.3.3 Study Group Demographic Summary 

The gender and ethnic distribution of subjects remained fairly consistent between Year 3 and 
previous years. There was only modest attrition and no new subjects were added to the sample. 
Demographic details are included in Appendix 1-E. 

4.4 Data Collection 
4.4.1 Methods 

The methods for data collection in Year 3 followed similar protocols as in previous years. 
Informed consent was obtained from each participant before any surveys, interviews, or 
observations were conducted. Copies of representative consent forms are included in Appendix 
1-D. 


Minor changes were made in the interview guides and are reflected in an updated interviewer 
manual (Appendix 3-C). The spring survey was modified to include questions that had arisen as 
a result of the dozen or so exit interviews completed prior to that point. To prevent the survey 
from becoming too lengthy, several items deemed less central to the APS research were dropped 
from the survey in order to accommodate the new questions (see Appendix 4- A). 

4.4.2 Summary of Data Collected 

Table 4.1 summarizes the data collected at the four core schools during Year 3. As in previous 
years, academic transcripts were collected but are not reported in this table. 


36 


APS Research Processes and Procedures 

May 2010 



Table 4.1 Data collection summary by school for Year 3 (2005-2006) 



Surveys 

Structured Interviews 

Semi-structured Ethnographic 
interviews 

Ethno Obs 

Exit 

interv 


Dates 

# 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'v 

Dates 

# 

Length 

(min.) 

# 

Eng. 

Act'y 

# 

Hours 

spent 

# 

TPub 

Fall 

2005 

38 











2 

Spring 

2006 

38 

3/6- 

5/24/06 

22 

n/a 

22 

3/9- 

4/20/06 

15 

65-168 

15 

0 

0 

1 

UPri 

Fall 

2005 

32 











2 

Spring 

2006 

32 

4/10- 

5/25/06 

18 

25-55 

17 

4/13- 

5/24/06 

12 

60-120 

11 

12 

24 

1 

SPri 

Fall 

2005 

40 












Spring 

2006 

40 

3/23- 

6/26/06 

17 

26-74 

17 

3/26- 

5/25 

15 

87-209 

12 

3 

4 

3 

LPub 

Fall 

2005 

34 









20 

60 

6 

Spring 

2006 

34 

4/24- 

5/20/06 

19 

60-90 

19 

Apr-May 

'06 

13 

90-150 

13 

40 

120 

0 


As mentioned earlier, one SPri participant who had completed an exit interview in Year 2 
returned to engineering. A "return to engineering" protocol was developed, and administered to 
this student at the same time as his structured interview. 

4.4.3 Cross-sectional Cohort 

As a result of collaborative relationships between APS researchers and colleagues at the Large 
Midwestern Public University (LMPub), LMPub researchers adopted the PIE survey for use with 
a cohort of engineering students there. This group became the Cross-sectional Cohort. Although 
the LMPub project was not formally part of APS, the two groups maintained a cooperative and 
mutually beneficial relationship. 

The goal for the Cross-sectional Cohort was to recruit 40 students from each undergraduate class 
(freshmen, sophomore, junior, senior), for a total of 160 participants. Students were recruited via 
e-mail, using enrollment lists from the engineering school. Rather than following students 
longitudinally, LMPub students were offered two discrete opportunities to participate: fall 2005 
and spring 2006. The fall survey included only those students who had entered LMPub as 
freshmen. The spring survey was offered to additional students including transfer students. 
Participating students received a gift card at the LMPub book store in the amount of $15 for one 
survey and $25 for both (fall 2005 and spring 2006). 

LMPub researchers customized the APS survey to reflect majors at LMPub. Two questions 
directed at transfer students were added to the spring survey (Appendix 4-E). Otherwise, the 
LMPub protocols were very similar to those used for Longitudinal schools during Year 3. 
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As already noted, the main differences between the APS Longitudinal Cohort and LMPub Cross- 
sectional Cohort were that the sampling at LMPub was cross-sectional rather than longitudinal, 
and it allowed for the inclusion of transfer students. 

Figures 4.2 and 4.3 summarize the gender and ethnicity distributions of the Cross-sectional 
Cohort. 

Figure 4.2 Cross-sectional Cohort gender distributions 



Fall 05 (non- Spring 06 (non- Spring 06 
transfers) transfers) Transfers 


Figure 4.3 Cross-sectional Cohort ethnicity distributions 

Note; Some respondents chose not to provide ethnicity data, so totals may not equal 100%. 
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4.5 Notes and Reflections on Year 3 

4.5.1 Difficulty Securing Exit Interviews 

Obtaining exit interviews with participants who declared non-engineering majors proved difficult 
because these subjects either lost interest in the study or failed to provide alternate contact 
information, especially if they left the university. 

4.5.2 Students Absent from Campus 

Several SPri students were not present on campus during Year 3. One took a term off for 
personal reasons, and another took the entire academic year off to work. Both students were able 
to complete their surveys and interviews as usual. Four other SPri students went abroad during 
the spring term and completed surveys remotely; two of them completed their interviews before 
leaving and two received semi-structured ethnographic interviews by phone while they were 
away. In sum, data collection was not impacted by the absences. 

4.5.3 Structured Interview Transcriptions 

Structured interviews were transcribed as in previous years of the study. However, the digital 
versions of the transcriptions were lost and had to be recreated by scanning working copies in 
paper version. 
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5 Research Year 4: Fall 2006-Spring 2007 

5.1 Year 4 Tasks and Goals 

The main tasks for Year 4 were to: 

• Administer one survey (spring) to all participants in the Longitudinal Cohort 

• Conduct informal conversations (fall-winter) and semi-structured ethnographic 
interviews (spring) with students in the Medium and High Contact Groups (Longitudinal 
Cohort) 

• Conduct exit interviews with Longitudinal Cohort students who declared majors other 
than engineering 

• Intensify efforts to analyze data and report findings 

• Conduct Workplace Cohort interviews (see Section 6) 

• Recruit Broader Core Sample participants and administer survey (see Section 7) 

• Plan for Broader National Sample and begin recruiting schools (see Section 8) 

5.2 Changes to Longitudinal Cohort Procedures 

5.2.1 Extended Data Collection 

As previously mentioned, supplemental NSF funds made it possible to extend Longitudinal 
Cohort data collection to Year 4. All Longitudinal Cohort participants received one survey 
(spring 2007). Students in the High and Medium Contact Groups received semi- structured 
ethnographic interviews and engineering design tasks in spring 2007, as well as informal 
conversations during the preceding fall and winter months. Low Contact participants did not 
receive structured interviews in Year 4. 

NOTE: Structured interviews were conducted at UPri but not as part of APS activities. 

5.2.2 Informal Conversations 

Informal conversations allowed researchers to check in with subjects as needed to stay abreast of 
changes in student status. Such conversations occurred as needed in prior years of the study but 
were not recorded in the APS database. In Year 4, researchers had the option of uploading 
informal conversations to the database. 

Informal conversations were arranged with students in the Medium and High Contact Groups, 
and generally focused on the question, "How are things going for you at this point?" The goal 
was to hear from students about their classes, feelings about engineering, life at their institution, 
and life in general. 

5.3 Data Collection 
5.3.1 Methods 

The data collection methods in Year 4 followed similar protocols as in previous years. As 
already mentioned, informal conversations were just that: loosely defined one-to-one 
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conversations about how things were going for students in the Medium and High Contact 
Groups. 

As in previous years, informed consent was obtained from each participant before any data were 
collected. 

5.3.2 Summary of data collected 

Table 5.1 summarizes Longitudinal Cohort data collection in Year 4. 

Table 5.1 Data collection summary by school for Year 4 (2006-2007) 


Shaded cells indicate time periods when no data collection of that type was scheduled or conducted. 



Surveys 

Informal 

Conversations 

Unstructured Interviews 

Exit 

Interv 

Acad 

Trans 


Dates 

# 

Dates 

# 

Dates 

# 

Length 

(min.) 

# Eng. 
Tasks 

# 

# 

TPub 



10/12- 

10/19/07 

8 





0 


Spring 2007 

35 



3/20- 

4/25/2007 

12 

69-192 

12 

1 

34 

UPri* 



... 

0 





0 


Spring 2007 

21 



4/6- 

5/4/2007 

11 

25-92 

11 

0 

22 

SPri 



10/13/06- 

1/19/07 

14 





0 


Spring 2007 

39 

3/15-4/8/07 

11 

5/4- 

6/5/2007'' 

16 

91-289 

16 

0 

38 

LPub 



10/2- 

1 0/30/2006’" 

13 





0 


Spring 2007 

31 



4/16- 

5/23/2007 

13 

120 

13 

0 

31 

Totai 


126 


46 


52 


52 

1 

125 


* UPri did not conduct informal conversations. 


One of the informal conversations at LPub was conducted on January 22, 2007. 

^ One of the unstructured interviews at SPri was initiated early (March 26, 2007) because the student was going 
overseas. 

5.4 Data Analysis 

A workshop in September 2006 brought APS researchers together to stimulate discussion of 
possible analyses across instruments and institutions. The meeting resulted in a number of papers 
reporting cross-instrument findings. Method- specific analyses continued to generate findings that 
were reported in papers and presentations. (See http://www.engr.washington.edu/caee/ for an up- 
to-date listing of papers and reports from the APS research.) 

5.4.1 Data Sets Used for Analysis 

The overall APS data set consists of data from surveys, interviews, engineering design tasks, 
ethnographic observations and academic transcripts. A detailed student-by-student accounting of 
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all longitudinal data (Years 1-4) is available from the APS database, as are the total numbers of 
students for whom data is available in each dataset, by year and by institution. 

The portions of this overall data set used in a particular analysis vary depending on the focus of 
the analysis. For example, one analysis may focus on data associated with a particular method, 
whereas another may focus on data associated with a particular study group (such as high contact 
vs. low contact subjects). As such, the numbers of APS subjects reported in various documents 
and papers may differ. 

5.4.2 Processing of Academic Transcript Data 

Following the completion of the spring 2007 term, the four core institutions submitted 
cumulative academic transcripts (i.e., 2003-2007) for longitudinal study participants. Academic 
transcripts were collected in previous years also, but the final collection was used for analysis. 

Academic transcripts were processed to normalize the data between institutions. 

• At the two schools (SPri and LPub) operating on the quarter system, credit hours were 
converted to semester units by multiplying by a factor of two-thirds (i.e., 3 quarter units = 
2 semester units.) 

• All grades were mapped to a 5-point scale (A, B, C, D, F). 

o TPub and UPri both use a 5-point grading system so no conversions were 
necessary. 

o At SPri, which issues plus/minus grades (A-i-, A-, B-i-, B-, etc.) in addition to 
straight letter grades (A, B, C, D), the plusses and minuses were dropped. So A-i- 
and A- converted to A; B-i- and B- converted to B; and so on. SPri's grade of NP 
(not passed) was converted to F. 

o LPub uses a 0 to 40 grading scale. These grades were converted to the 5-point 
scale as follows: 35-40=A, 25-34=B, 15-24=C, 7-14=D, and 0-6=F. 

• Each course listed on transcripts was categorized as: 

o Engineering (eng) 
o Science, medicine (sci/med) 
o Humanities, social sciences, fine arts (hum/ss/fa) 
o Math, computer science (math/cs) 

o Physical education, freshman success seminar, etc. (other) 

• Courses were also assigned a design code whereby: 0 indicates no design elements; 1 
means non-engineering design (e.g., art classes); 2 denotes some engineering design 
(such as a class project); and 3 indicates a major focus on engineering design (e.g., 
courses defined by a single class project.) 

Academic transcripts were used to determine persistence in an engineering major, and the 
semester during which non-persisters declared non-engineering majors. 

5.4.3 Engineering Persistence Data 

Persistence in an engineering major was determined from academic transcripts; persisters were 
those whose transcript listed a major or minor in an engineering field at the time of graduation. 
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or in spring 2007, if they had not yet graduated. Non-persisters were those who declared a non- 
engineering major after initially intending to study engineering. 

The question of persistence in engineering was a central focus of the APS. Tables 5.2 and 5.3 
provide a summary of engineering persistence among APS participants at the end of the study 
period. The categories are defined as follows: 

• Persisters - students who entered the study with interest in engineering and whose 
academic transcripts at the end of the study period (spring 2007 or upon graduation) 
listed an engineering major or minor. 

• Non-persisters - students who entered the study with interest in engineering but 
subsequently declared a non-engineering major. Some non-persisters continued in the 
study through Year 4 while others chose not to. 

• Other - students who did not meet the criteria for either persister or non-persister. 

• Lost to follow-up - students who left the study without sufficient data to determine 
persistence status. 

Table 5.2 Persisters at end of Year 4: Subjects who enrolled in Year 1 (n=160) and Year 2 
(n=18) 



Persisters 

Non- 

persisters 

Other 

Lost to 
Foiiow-up 

TPub 

33 

8 

2* 

2 

UPri 

29 

8 

0 

15 

SPri 

27 

13 

■j ** 

0 

LPub 

32 

6 

0 

2 

Total 

121 

35 

3 

19 


Table 5.3 Persisters in Longitudinal Cohort at end of Year 4: Only subjects who enrolled in 
Year 1 (n=160) 



Persisters 

Non- 

persisters 

Other 

Lost to 
Foiiow-up 

TPub 

29 

8 

1* 

2 

UPri 

20 

7 

0 

13 

SPri 

26 

13 

■j ** 

0 

LPub 

32 

6 

0 

2 

Total 

107 

34 

2 

17 


* Non-engineering students from the beginning of the study. 

** Student left school to work in high-tech start-up company, with intention of returning to school eventually. He 
continued to participate in APS surveys and interviews but did not return to school during the study period. 

Various APS analysis teams may define their data sets differently, so the numbers in Figure 5.2 
may not be consistent across all APS papers and reports. 

5.5 Notes and Reflections on Year 4 
5.5.1 Attrition in Numbers 

The Longitudinal study group was essentially unchanged from Year 3. However, attrition is 
apparent in the numbers of completed surveys and interviews for Year 4. Reasons include 
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waning interest among subjects, early graduation, and permanent or temporary leaves from 
school. These factors contribute to ambiguities about the number of students completing the 
study. As mentioned in the previous section of this chapter, different analysis teams may report 
different numbers depending on how they choose to define their datasets. 
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6 


Broader Core Sample 


6.1 Research Goals and Description 

The goal of research using the Broader Core Sample was to confirm that findings from the 
Longitudinal Cohort were representative of the larger undergraduate engineering population at 
the four Longitudinal Cohort institutions: Technical Public Institution, Urban Private University, 
Suburban Private University, and Large Public University. The Broader Core Sample research 
pursued key questions that emerged from the longitudinal work: 

1. How are students who persist in an engineering major similar and different from students 
who do not persist? 

2. How do experiences of engineering students change as they progress through their 
undergraduate careers as freshmen, sophomore, juniors and seniors? 

3. How do men and women differ in their experiences in engineering education? 

The sole data collection tool was an online survey, the Academic Pathways of People Learning 
Engineering Survey (APPLES), derived from the PIE survey instrument administered to 
Eongitudinal Cohort. A major consideration in the design of APPEES was to reduce the length of 
the survey while retaining comparability of the two instruments so that generalizability of PIE 
findings could be tested. 

Because the Broader Core Sample and Broader National Sample both involved cross-sectional 
research to validate longitudinal findings using the APPEES instrument, these two cohorts are 
sometimes referred to as APPEES 1 and APPEES2 respectively, although the survey instruments 
were not identical (see Chen et al. 2008). 

Broader Sample research activities were carried out by the APS researchers at Stanford. 

6.2 Institutional Review Board (IRB) Approval 

Eocal IRB approval for (non-medical) human subjects research was required for each of the four 
institutions participating in the Broader Core Sample because each school had active APS 
researchers. The Stanford University APS team took the lead in drafting the protocols and 
coordinating the IRB submissions. UPri incorporated TPub into their IRB application since TPub 
doesn't have an Institutional Review Board. 

As with the PIE survey, design of APPEES was guided by universal IRB requirements: subjects 
were not required to answer survey questions, and all students had access to the incentive. In 
other words, students could claim the incentive without completing the survey, and regardless of 
whether they were part of the targeted student groups. However, the survey was designed so that 
researchers could identify such submissions and exclude them from data analysis. 
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6.3 Recruitment 

6.3.1 Sampling and Stratification 

To address the key research questions for the Broader Core Sample, recruitment efforts targeted 
three groups of undergraduate students: 

• Engineering students who had declared an engineering major or committed to engineering 
programs at their universities 

• Pre-engineering students who intended to declare an engineering major 

• Non-persister students who were interested in engineering at one time but had since 
decided to pursue another field of study. 

Persistence, along with academic level (freshman, sophomore, junior, senior) and gender, 
constituted the primary strata for setting recruitment targets. Secondary strata included 
enrollment status (part-time or full-time), transfer status, ethnicity and citizenship. Each stratum 
required 10 to 25 subjects per school to run statistical t-tests. Recruitment targets were set 
accordingly, as shown in Figure 6.1. 

A response rate as high as 30 percent per school was anticipated, based on the National Survey 
of Student Engagement (NSSE), a similar survey in terms of length and content, but one which 
offers no incentive for participation. 

6.3.2 Recruitment 

Each of the participating institutions named a coordinator for recruitment efforts at their school. 
Coordinators secured local IRB approvals with assistance from Stanford, and oversaw local 
recruitment during the survey deployment. 

A combination of approaches was used to recruit students for the Broader Core Sample. These 
included: 

• Posters hung in locations frequented by undergraduate engineering students (Appendix 2-C) 

• Emails to students from their engineering dean, using targeted distribution lists (Appendix 2- 
D) 

• Advertisements in the student newspaper and, for one institution, a directed advertisement 
on a social networking site. 

All recruitment materials and communications carried a red apple logo designed for the study, 
and provided the dates and URE for accessing the survey. Each institution had its own APPEE 
Survey URE that included the institution’s name to enhance credibility and avoid being mistaken 
for spam. 

Recruitment Plans were developed jointly with the local campus coordinator and a liaison from 
the research team at Stanford (see Appendix 2-B). On the recruitment planning form. Plan A 
shows the initial strategies for the institution; Plan B delineates targeted strategies in case 
responses in any stratum lagged. Campus coordinators implemented Plan B only if they 
perceived one or more of their strata to be lagging during survey deployment. 
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6.3.3 Incentive for Participation 

To encourage participation, a $4 electronic payment through PayPal was offered to every 
individual who filled out the survey. The $4 payment best met budgetary and other criteria, 
including: 

• Broad student appeal 

• Available online almost immediately 

• Scalable with minimal logistical effort for large or small, geographically distributed 
audiences 

• Redeemable without compromising student confidentiality 

• Payments could be tracked to meet university disbursement requirements 

• Unclaimed incentives could be returned to the project after a period of time. 

Other incentive options that were considered included a chance at a raffle prize and various gift 
card options (books, coffee, sandwiches, music, etc.). 

6.4 Data Collection 

6.4.1 The Survey Instrument 

The APPLE Survey is comprised of a focused subset of questions from the PIE survey used with 
the Eongitudinal Cohort. The instrument underwent two rounds of pilot testing to ensure its 
effectiveness for a single cross-sectional administration that would include non-engineering 
majors. The first round of piloting involved 10 researchers and graduate students who were 
associated with the project. The goal was to refine the survey questions for clarity and identify 
questions that could be eliminated. The second round of pilot testing involved 58 undergraduate 
students from five external institutions (i.e., schools not affiliated with APS). This round of 
testing was aimed at paring down the survey so it would take only ten minutes to complete, as 
compared to the 30-minute completion time for the PIE survey. 

The resulting APPEESl instrument consisted of 52 questions, representing 26 constructs (also 
called variables) and five demographic items. These constructs are described in detail in 
Appendix 4-B, which also maps the three APS survey instruments (PIE, APPEESl /Broader Core 
Sample, APPEES 2/Broader National Sample) against one another. The survey is included in 
Appendix 4-C. 

6.4.2 Survey Administration 

APPEESl was deployed at all four campuses from April 2 through April 9, 2007. The 
deployment period was extended two-and-a-half weeks (until April 27) at Urban Private 
University in an attempt to better meet strata targets with its small pool of engineering students 
and infrastructure constraints in reaching students by email. 

Response to the online deployment was monitored in real time, allowing for remedial measures 
to be taken to attract survey-takers in strata that might be lagging. Response reports were sent 
daily to institution liaisons in the late afternoon. A sample report is included in Appendix 5-A. 
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6.4.3 Summary of Data Collected 

There were a total of 914 survey responses from the four schools, of which 842 were included 
for analysis. The 72 submissions that were omitted were determined to be ineligible (i.e., 
submitted by graduate students or students from non-participating universities) or fraudulent. 
Fraud was defined as "blind clicking" or otherwise attempting to claim the incentive without 
filling out the survey, and represented approximately 3 percent of submissions. Blind clicking 
was assumed for submissions with completion times of less than five minutes, a cut-off derived 
from pilot testing. 

Table 6.1 provides a summary of the cleaned (eligible) data set by strata and by institution. 


Table 6.1. Summary of Broader Core Sample Response Targets and Eligible Responses 


Strata 

Target per 
school 

TPub 

UPri 

SPri 

LPub 

Totais 

All 

140 

239 

67 

217 

318 

842 

Freshmen* 

25 

67 

14 

77 

31 

189 

Sophomore* 

25 

60 

15 

33 

49 

157 

Juniors* 

25 

48 

14 

62 

122 

246 

Seniors* 

25 

56 

20 

38 

90 

204 

5th Yr Senior or more 

(not set) 

8 

2 

6 

25 

41 

Transfer students 

10 

24 

9 

8 

74 

115 

Returning or Non-traditionai student 

(not set) 

12 

2 

1 

17 

32 

Non-persisters 

25 

12 

13 

36 

28 

89 

Maie students* 

70 

157 

35 

124 

223 

539 

Femaie students* 

25 

81 

31 

91 

93 

296 

Ethnic minority students^ 

25 

21 

57 

37 

19 

134 

International students^ 

25 

11 

23 

22 

45 

101 

Part-time students 

10 

5 

0 

1 

4 

10 


* Primary strata 

* Ethnic groups traditionally underrepresented in undergraduate engineering programs in the U.S., including Native 
American, African American/Black, Hispanic/Latino 

Students who do not hold U.S. citizenship 

Of the total 914 survey responses, 137 declined the incentive. Ultimately, 562 subjects (72 
percent of subjects who accepted the incentive) collected the $4 incentive. 

Response rates, i.e., the number of respondents relative to the undergraduate engineering 
enrollment, were as follows: 


TPub 

10% 

UPri 

12% 

SPri 

34% 

LPub 

21% 

Overall 

17% 
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6.5 Notes and Reflections 


6.5.1 Pilot Testing 

Pilot testing proved to be an invaluable resource for refining the survey instrument. For this 
reason, piloting of the APPLES instrument was extended to two rounds despite short timelines. 
In addition to providing feedback about readability and comprehension, pilot data helped guide 
the difficult decisions about which items and constructs to keep or delete so respondents could 
complete the survey in ten minutes. Comparable surveys, including the National Survey of 
Student Engagement (NSEE), strive for a ten-minute take-time to maximize response rates. Pilot 
testing also helped determine a minimum completion time (five minutes) below which "blind 
clicking" could be reasonably assumed. 

6.5.2 IRB Approvals 

Obtaining IRB approval took longer than expected at all institutions, and requirements were not 
consistent across institutions. Eor example, EPub's IRB called for a confidentiality disclaimer 
statement and UPri's IRB wanted wording changes in the recruiting materials. Although 
researchers had hoped to offer survey-takers the option of online or paper surveys, 
confidentiality requirements made this option unfeasible, particularly when it came to claiming 
the incentive. 

6.5.3 Recruitment Challenges 

None of the participating institutions met their targets for all strata. Eor the most part, this had to 
do with the particular characteristics of the schools (e.g., difficulties locating non-persisters at 
TPub, UPri and SPri and weaknesses in the campus email system at UPri). 

Reaching non-persisters required special recruitment efforts. Eor example, at SPri, Eongitudinal 
Cohort data suggested that students who had opted out of engineering commonly went into Math 
and Computational Sciences, Physics, Symbolic Systems and Economics. Therefore, the non- 
persister stratum target was successfully met by emailing students in these departments and 
asking them to participate in the survey. As with the recruitment email that went to engineering 
undergraduates, the email seeking non-persister respondents went out over the signature of a 
dean (Appendix 2-E). 

6.5.4 Repeat Claims for Incentive 

Researchers had considered limiting responses to one per IP address in order to discourage 
respondents from making repeat claims for the incentive. However, this would have excluded 
participants who may share a computer or use a computer lab. It turned out that a number of 
eligible survey responses from UPri came from a single IP address. 


Eor more information about APPEES design and development, see: 
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Donaldson, Krista M., Helen L. Chen, George Toye, Sheri D. Sheppard. 2007. Targeting 
Undergraduate Students Interested in Engineering for Surveys: Lessons from the Academic 
Pathways of People Learning Engineering Survey. Frontiers in Education Annual 
Conference, Milwaukee, WI, October 10-13, 2007. 

Chen, Helen L., Krista M. Donaldson, Ozgur Eris, Debbie Chachra, Gary Lichtenstein, Sheri 
D. Sheppard, George Toye. 2008. Erom PIE to APPLES: The Evolution of a Survey 
Instrument to Explore Engineering Student Pathways. American Society for Engineering 
Education Annual Conference, Pittsburg, PA, June 2008. 
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7 


Broader National Sample 


7.1 Research Goals and Description 

The goal of research with the Broader National Sample was to corroborate and extend findings 
from the Longitudinal Cohort and the Broader Core Sample with an expanded set of 
undergraduate engineering students and institutions nationwide. The sole data collection 
instrument was the APPLE survey, administered cross-sectionally during January to March 
2008. Power calculations indicated a minimum of 1,080 respondents from 18 institutions were 
required to ensure a nationally representative sample of undergraduate engineering students. 

7.2 The Research Team 

All aspects of research with the Broader National Sample were conducted by the Stanford APS 
research team. Each member of the Stanford team served as liaison to several (four to six) 
participating institutions. Eiaison duties vis-a-vis their assigned schools included: 

• Extending and following-up on invitations to participate 

• Keeping the schools appraised of study timelines and procedures 

• Working with each institution to devise and execute a recruitment plan appropriate to the 
school and its students 

• Monitoring the school's response rates during survey deployment, including creating and 
sending daily response reports (see sample response report in Appendix 5-A) 

• Serving as the primary contact person for and with the institution 

7.3 Institutional Review Board (IRB) Approval 

Umbrella IRB human subject approval was obtained from Stanford University and covered all 
participating students and institutions except for two institutions with APS researchers on staff. 
Those two institutions submitted applications to their respective IRBs, and both were approved. 
As it turned out, four other participating institutions voluntarily chose to obtain approval from 
their local IRB, and did so with assistance from the Stanford team. 

7.4 The Survey Instrument 

Based on results from the Broader Core Sample, the APPEESl instrument underwent minor 
modifications to: 

• Ensure that demographic questions were appropriate and detailed enough to capture 
institutional and respondent diversity 

• Improve the internal reliability of several constructs 

• Address intrinsic (psychological) motivation, which was mentioned in the open-ended 
comments from APPEES 1 respondents 

The resulting APPEES2 instrument was pilot tested with a sample of 52 undergraduate 
engineering students from three external institutions (i.e., schools not participating in or affiliated 
with APS research). 
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The survey constructs are described in detail in Appendix 4-B, which also maps the three APS 
survey instruments (PIE, APPLES 1 /Broader Core Sample, APPLES 2/Broader National Sample) 
against one another. Eor a more in-depth review of the APPLES instrument, see the technical 
report CALL TR-09-02, Exploring the Engineering Student Experience: Eindings from the 
Academic Pathways of People Learning Engineering Survey (APPLES). 

7.5 Sampling Plan 

Schools were identified and invited to participate in the Broader National Sample according to 
stratification criteria that would yield a diverse pool of students for the survey. The institutional 
characteristics that drove the stratification process were: 

1 . Carnegie 2000 classification 

2. Student body ethnic composition, gender balance and enrollment status (full-time vs. part- 
time) 

3. Institution size, type (public vs. private), geographic location, presence/absence of religious 
affiliation and number of transfer students 

The number of institutions was driven by the calculated number of respondents (1,080) needed 
for statistically meaningful results. A slightly higher number of institutions were recruited to 
provide a cushion in case of last-minute withdrawals. Tables 7.1 and 7.2 summarize the 
institutional sampling stratifications, as well as the numbers of institutions that were ultimately 
recruited in each stratum. 


Table 7.1. Summary of Primary and Secondary Stratification Characteristics 


Type of Institution 

Required 

Participated 

Primary Stratifications 


Doctoral/Research - Extensive 

5 

7 


Doctoral/Research - Intensive 

2 

4 


Specialized Institutions - Engineering 

2 

3 


Master’s Colleges and Universities I 

2 

3 


Specialized Institutions - Other 

1 

0 


Baccalaureate Colleges - General 

1 

2 


Baccalaureate Colleges - Liberal Arts 

1 

2 

Secondary Stratifications 


Historically Black Colleges and 
Universities 

1 

2 


Hispanic-Serving Institutions 

1 

2 


Single-Gender Institutions 

1 

1 


Part-Time Student Population > 30% 

1 

4 


Recruiting Redundancy 

3-7* 

3 


TOTAL 

21-25 

21 


*We estimated we needed to recruit 3-7 additional institutions should one or more 
institutions be unable to participate in APPLES late in the process. 
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Table 7.2. Summary of Tertiary Stratification Characteristics 


Tertiary Stratification Considerations 

National Picture (2007)* 

Participating 

Institutions 

Institution size (based on enrollments) 

Large = 54% 
Medium = 43% 
Small = 3% 

Large = 33% (7) 
Medium = 38% (8) 
Small = 29% (6) 

Geographic diversity 


17 states represented 

Funding type 

Public = 63% 
Private = 37% 

Public = 67% (14) 
Private = 33% (7) 

Religious affiliation 

14% of institutions 
4% of population 

5% (1 institution) 

Transfer student population 

(Information not available) 

Two 3+2 completion 
institutions 


*Percentage of national sample of 319 institutions 


7.6 Institutional Selection and Recruitment 

Guided by the sampling plan, researchers strategically selected schools to invite into the Broader 
National Sample research. Invited schools were required to have at least one ABET-accredited 
engineering major in addition to the characteristics laid out in the sampling plan. A total of 319 
institutions met these criteria. Furthermore, schools where APS researchers had personal contacts 
were favored in case we encountered institutional hurdles that an insider might help us 
understand and overcome. As previously stated, researchers sought to err on the conservative 
side by inviting more schools than were required by the sampling plan. 

Beginning in spring 2007, the research team mailed letters of invitation to the engineering deans 
at 25 institutions. Stanford liaisons followed-up by telephone and e-mail to answer questions and 
secure commitment to participate. In addition, researchers held a special session at the American 
Society of Engineering Education annual conference in June 2007 to further describe the 
research, answer questions and secure commitments from the targeted institutions. 

As an incentive to participate, each institution received a report summarizing the data submitted 
by its students relative to those from the other participating schools. 

Twenty-one institutions accepted the invitation to participate in the Broader National Sample, as 
illustrated in Figures 7.1 and 7.2. Invitations were extended to three different military academies, 
none of which accepted, leaving us without representation from Carnegie 2000 classification 
Specialized Institutions — Other. 

Participating schools were asked to appoint a local coordinator to assist researchers in 
understanding local campus culture, provide institutional data (such as school calendars and 
enrollment figures), and plan and implement local student recruitment. Appendix 2-B includes 
sample forms that were used to plan recruitment efforts. 
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Number of Undergraduate Engineering Students 


7.7 Subject Recruitment 

7.7.1 Diversity Goals and Strata Targets 

In keeping with our research goals, student recruitment was planned to ensure a diversity of 
participants, including over-sampling of specific groups (strata). The strata, in order of 
importance, were defined as follows: 

• Primary strata: academic level (freshmen, sophomores, juniors, seniors), engineering 
persisters/non-persisters, and men/women 

• Secondary strata: ethnic minority and international students 

• Tertiary strata: part-time and transfer students 

Ethnic minorities included those traditionally underrepresented in the undergraduate engineering 
population nationally: African Americans, Latinos, and Native Americans. International students 
were defined as those not holding U.S. citizenship. 

Strata recruitment targets were set for each participating institution by visually-binning 
according to their undergraduate engineering enrollment: small (less than 500 students), 
medium-small (500-1000 students), medium-large (1000-3000 students), and large (more than 
3000 students) (Figure 7.3). 


Figure 7.3. Strata Targets by Institutional Undergraduate Engineering Enrollment 


7000 


6000 


5000 


4000 


3000 


500 < MEDIUM-SMALL < 1,000 
Primary 20 
Non-persisters and 
secondary: 15 
Tertiary 10 


1,000 < MEDIUM-LARGE < 3,000 
Primary: 25 

Non-persisters and secondary: 20 
Tertiary: 10 



SMALL < 500 
Primary strata: 10 
subjects 
Other strata: 10 
subjects (as 
appropriate) 


.-S' .-S' .-S- .-S- 

S<? S<? S<? S<? # s?®’ nS? nS? 


LARGE = >3,000 
Primary: 50 
Non-persisters and 
secondary : 30 


Tertiary: 20 s Jbjects 


Institution 


The local coordinator and Stanford liaison adjusted strata targets depending on the student 
composition of the school. For example, if a small school had fewer than ten international 


54 


APS Research Processes and Procedures 

May 2010 


students, this stratum (and target) was eliminated. Similarly, if a school had a large proportion of 
Latino students, the target for ethnic minorities was increased. 


7. 7.2 Incentive for Participation 

As in the Broader Core Sample, students were offered a $4 electronic payment through Pay Pal 
for their participation in the Broader National Sample. 

7. 7.3 Recruitment Methods 

The main methods for recruiting students to take the APPLE survey were: 

• Email messages to undergraduate engineering students from a senior administrator (such 
as a dean). Coordinators were encouraged to send the email on the first day of survey 
deployment. 

• Posters customized to the institution. Posters typically went up the weekend before 
deployment at locations frequented by engineering students. (See Appendix 2-C for 
sample poster.) 

A third recruitment method, advertisement on a popular social networking site, was used as 
needed to bolster response rates mid-deployment. This step was required at seven (33%) of the 
schools. 

In November 2007, prior to deployment, local coordinators developed strategic recruitment plans 
for strata that may be more difficult to fill, such as non-persister, ethnic minority or part-time 
students. Strategic recruitment was implemented based on the daily response reports prepared by 
Stanford liaisons during the week of survey deployment. A sample response report is shown in 
Appendix 5-A. 

7.8 Survey Deployment 
7.8.1 Deployment Procedures 

APPEES deployments for the Broader National Sample were scheduled to last five days 
(Monday through Eriday), consistent with response patterns from the Broader Core Sample 
deployment. Participating institutions could choose from three deployment weeks, ranging from 
late January to late Eebruary 2008. A fourth deployment week in March 2008 was added to 
accommodate two institutions that were unable to schedule deployment earlier. 

The survey was "turned on" at 12:01 AM on Monday, and "turned off" at 1 1:49 PM on Eriday. 

At nine institutions where response rates were low and strata targets were unmet, the survey was 
kept live for up to one week longer. 

Participants accessed the survey using a URE that included the school name or abbreviation 
(e.g., schoolname.applesurvey.org). The school reference was incorporated in the UREs to 
enhance the credibility of the survey and encourage participation. 
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7.8.2 Data Collection Summary 

Table 7.4 summarizes the data collected by stratum. 


Table 7.4. Summary of Eligible APPLES Responses by Stratum 


UG Eng APPLES 


School 

Enrollm't 

Responses 

47 

Fr 

So 

Jr 

1 

99 

(47%) 

95 

13 

11 

13 

2 

251 

(38%) 

116 

19 

28 

22 

3 

286 

(41%) 

87 

29 

21 

37 

4 

310 

(28%) 

84 

24 

22 

22 

5 

546 

(1 5%) 
155 

25 

17 

20 

6 

634 

(24%) 

54 

29 

35 

43 

7 

666 

(8%) 

131 

10 

13 

13 

8 

823 

(16%) 

153 

11 

25 

38 

9 

874 

(1 8%) 
57 

30 

14 

41 

10 

896 

(6%) 

160 

12 

18 

14 

11 

1,397 

(11%) 

261 

41 

39 

49 

12 

1,405 

(19%) 

101 

64 

71 

68 

13 

1,517 

(7%) 

136 

21 

24 

27 

14 

1,563 

(9%) 

635 

48 

33 

28 

15 

1,623 

(39%) 

361 

177 

104 

148 

16 

1,674 

(22%) 

242 

71 

84 

90 

17 

1,723 

(14%) 

99 

18 

96 

83 

18 

2,245 

(4%) 

391 

26 

6 

29 

19 

2,290 

(17%) 

445 

57 

67 

99 

20 

5,930 

(7%) 

456 

97 

127 

120 

21 

6,591 

(7%) 

4,266 

115 

112 

117 

Totals 

33,343 

(13%) 

937 

967 

1,121 
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Sr+5th 
Yr Sr 

M 

F 

Ethnic 

Minor'y 

Int'l 

Part- 

time 

10 

0 

47 

5 

6 

0 

26 

75 

20 

7 

10 

1 

29 

64 

51 

5 

7 

0 

19 

41 

45 

2 

4 

0 

22 

71 

12 

9 

9 

0 

47 

101 

54 

122 

39 

8 

18 

42 

12 

52 

1 

2 

54 

92 

35 

6 

14 

14 

65 

109 

44 

43 

38 

18 

12 

46 

11 

1 

3 

1 

31 

89 

70 

7 

12 

0 

58 

148 

113 

26 

40 

0 

28 

70 

28 

42 

48 

6 

26 

80 

54 

112 

10 

2 

203 

481 

146 

19 

63 

15 

114 

268 

91 

44 

36 

26 

45 

110 

131 

31 

19 

0 

37 

79 

18 

6 

17 

5 

168 

303 

83 

26 

17 

3 

101 

260 

183 

25 

90 

1 

111 

261 

194 

62 

39 

7 

1,224 

2,790 

1,442 

652 

522 

109 
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There were 4,597 eligible (non-fraudulent) responses in the Broader National Sample, which was 
reduced to 4,266 after data cleaning. Approximately 3,900 (85%) of eligible respondents 
indicated they wished to claim the incentive, of whom 2,958 (64% of eligible respondents) 
actually did so. 

7.9 Notes and Reflections 

7.9.1 Controlling Incentive Outlays 

Prior to deployment, the research team considered the financial burden of issuing incentives in 
case of an unexpectedly high response rate, such as the 30 percent that the NSSE surveys gamer. 
This concern was addressed in several ways: 

• Emphasis was placed on strategic recruitment aimed at meeting strata targets, with little 
to be gained (statistically) from grossly exceeding targets. 

• Daily monitoring of responses would allow researchers to know if numbers were getting 
too large, in which case the survey could be turned off once strata targets were 
comfortably met. 

Eurthermore, approximately 61 percent of respondents in the Broader Core Sample ended up 
collecting their incentives, and there was no reason to expect differently in the Broader National 
Sample. 

As it turned out, response rates were similar in the two Samples (17% for the Broader Core 
Sample and 14% for the Broader National Sample), as were incentive collection rates (61% vs. 
64%). 


7.9.2 Fraudulent Responses 

There were two cases of attempted large-scale fraud, defined as a large number of ineligible 
submissions. The first case was detected by an unexplained surge in responses halfway through 
the deployment week. This surge was traced to two "free money"-type websites to which the 
APPEES URE had been forwarded. The second case of fraud was the result of two individuals 
repeatedly taking the survey, one taking it 14 times and the other 38 times. Multiple submissions 
were detected using a combination of IP tracking and timing data. As in the Broader Core 
Sample, the minimum time for taking the survey without blindly clicking through fell at the five- 
minute mark. Submissions that were clearly fraudulent were removed from the data set. 

7.9.3 Recruitment Challenges 

As with the Broader Core Sample, researchers experienced difficulty recruiting non-persisters, 
transfer students and part-time students. Non-persisters were most successfully recruited via 
email sent to technical non-engineering departments such as Physics. However, not all 
institutions had such majors and some institutions faced internal constraints in contacting 
students outside engineering. Transfer students were most easily recruited at large public 
institutions and those that enrolled 3-1-2 students (i.e., students who completed three years at 
another institution in order to transfer to complete their last two years in engineering). 


APS Research Processes and Procedures 
May 2010 


57 



7.9.4 Lessons Learned 

Despite the experience gained from the Broader Core Sample, researchers learned additional 
lessons about survey recruitment and deployment from the Broader National Sample. 

• There were advantages to using institution-specific URLs for accessing the survey, 
including more thoroughly protecting the identity of participating institutions from each 
other; facilitating the tracking and remediation of technical problems; and allowing 
researchers to isolate large-scale fraud without impacting the larger deployment or data 
sets. 

• The last question on the survey, "Is there anything else you want to tell us that we didn't 
already cover?" was a rich source of data. Many students revealed passions, concerns and 
experiences that were not otherwise captured in the survey. 

• The amount of researcher attention required by the participating institutions varied. 
Researchers generally spent more time communicating with and assisting institutional 
coordinators than was anticipated. In at least one case, a researcher had to step in to 
locate and recruit students at a school that did not have experience or knowledge to do so 
themselves. 

• Similarly, managing incentive payments required more time and effort than anticipated, 
largely due to student requests for help and demands for payment. 

• Staggering deployments relieved considerable pressure on the research team to better 
address the needs of participating institutions, provide for last-minute survey extensions 
and resolve anomalies with incentive claims. 

For more information about survey development for the Broader National Sample/ APPLES 2 

please see: 

Donaldson, Krista, Helen Chen, George Toye, Mia Clark, and Sheri Sheppard. 2008. 

Scaling Up: Taking the Academic Pathways of People Learning Engineering Survey 

(APPLES) National. Presented at the 38th ASEE/ISEE Erontiers in Education Conference, 

Saratoga Springs, NY, October 22-25, 2008. 
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8 Workplace Cohort 

The goal of the Workplace Cohort research was to address research questions about the school- 
to-work transition of new professional engineers. Of particular interest were those aspects of 
becoming an engineer that are not taught or learned as part of academic training. By studying 
new engineers and their supervisors, researchers explored technical and social factors and .skills 
that contributed to a successful transition into the workplace. 

The original design of the Workplace Cohort consisted of 16 students (8 on each of two 
campuses) who had participated in the Longitudinal Cohort research. The plan was to follow 
these 16 students from the end of their junior year through the first two years post-B.S. 

However, this original design of the Workplace Cohort was expanded to eventually include over 
100 participants in three distinct studies. We call these sets of archived interviews: 

1) The Workplace Cohort: Researcher#! . These interviews were conducted by a CAEE 
research scientist at the University of Washington under the supervision of a CAEE Eead 
atUW. 

2) The Workplace Cohort: Researcher#2 . These interviews were conducted by a CAEE 
research scientist at the University of Minnesota (at the time a CAEE-funded PhD 
student), under the supervision of two CAEE Eeads at Stanford and University of 
Minnesota. 

3) The Workplace Cohort: Researcher#3 . These interviews were conducted by a CAEE 
research scientist at the University of Washington under the supervision of a CAEE Eead 
atUW. 

Each of these Workplace Cohorts is described below. 

8.1 Workplace Cohort: Researcher#! 

This dataset consists of eight interviews with engineering graduates from one of the APS core 
schools. All eight were Eongitudinal Cohort students, and completed their degrees in 2008. 
Three of the interviewees are female and five are male. The interviews were conducted between 
April 2008 and November 2008, and ranged in length from approximately 20 minutes to an hour. 
At the time of the interview, four of the engineering graduates were employed in medium to 
large sized engineering firms, three in small firms, and one was not in the workplace. The 
overall intent of these interviews was to explore the transition from engineering educational 
institutions to actual engineering workplaces. The topics of interest included: the use of 
mathematics in the workplace, differences in learning experience in school and work, and 
aspects of engineering education that were important in the workplace but not covered in school. 
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8.2 Workplace Cohort: Researcher#! 

This dataset consists of 96 transcripts of interviews of engineering graduates employed as 
engineers or engineering supervisors at one of four organizations (see Table 8.1 for descriptions 
of organizations). The data collection was mainly focused on understanding the socialization of 
newly graduated engineers who had been employed at the organization from 6-12 months, 
addressing such questions as: 

• How do newly hired engineers practice engineering in the workplace? 

• How do newly hired engineers learn the specific job requirements and social norms of 
the workplace? 

• What are the factors affecting how newly hired engineers begin practicing engineering in 
the organizational setting? 

The interview protocol (based on a critical incidence methodology) is contained in Appendix 3- 
E. The interviews were conducted between January and April 2007, and were nominally an hour 
in length. 


Table 8.1. Four organizations participated in the Workplace Cohort: Researcher #2 

Organization #1 — is a large, global vehicle manufacturing company. They had recently 
reorganized and began hiring new engineering talent primarily to develop new technologies. 
There was much flux in the organization and they had delegated new hire onboarding to the 
managers of work groups without a structured plan to follow. Therefore, each work group 
onboarded new hires according to the preferences and conditions of the work group — meaning 
that there was a high level of variance in the experiences of new hires — from good to bad. 

This dataset includes semi-structured interviews with 30 newly hired engineers and 6 supervisors 
conducted in January and February of 2007. The new hires had from 6 months to 18 months 
experience at their jobs. Each supervisor had at least one of these new hires in his or her work 
group. 

Organization #2 — is a large national food manufacturing company. They had a highly 
structured onboarding program including a rotational program for newly hired engineers. While 
new hires were assigned to different production plants, the experiences were relatively similar. 
New hires received detailed plans for meeting with others (developing relationships and 
networks) and the organization explicitly expected managers at all levels to meet with new hires 
and expected work groups to provide learning experiences to new hires. New hires also moved 
across three different engineering jobs and work groups during their rotations. This structured 
process enabled new hires to quickly learn the culture of the company and make important 
contacts early, as well as learn about different processes and functions in the company. 

This dataset includes semi-structured interviews with 1 8 newly hired engineers conducted in 
January through April of 2007. The new hires had from 6 months to 18 months experience at 
their jobs. 
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Organization #3 — is a smaller manufacturing company of computer components. They 
provided a structured and systematic onboarding process comprised of training courses and 
meeting various experts in the company. They did not have a rotational program and the 
experiences of new hires depended on the efforts of the work groups to help new hires learn and 
integrate into the organization. 

This dataset includes semi-structured interviews with 19 newly hired engineers and 4 supervisors 
in February of 2007. The new hires had from 6 months to 18 months experience at their jobs. 
Each supervisor had at least one of these new hires in his or her work group. 

Organization #4 — is a state-government agency for transportation. This organization had a 
highly structured, rotational program providing new hires with in-depth background information 
and the opportunities to rotate among various work groups/experiences. The experiences of new 
hires varied among work groups depending on the efforts of coworkers and managers to facilitate 
learning. 

This dataset includes semi-structured interviews with 19 newly hired engineers conducted in 
January through April of 2007. The new hires had from 6 months to 18 months experience at 
their jobs. 


8.3 Workplace Cohort: Researcher#3 

This dataset consists of seven audio files of interviews of recently graduated engineers (all were 
out of school for less than 12 months at the time of their interview). The companies at which 
these engineers work were identified through discussions with staff in engineering departments 
at one of the APS core schools as companies that recruit the school’s graduates. 

Five of these interviews were with individuals at a county public works department. These 
individuals were identified and recruited through a list provided by the department's director and 
the head of the training program for new engineers. 

One interview was conducted with an engineer at a state transportation agency who was selected 
from a list provided by a manager in the agency. 

One interview was conducted with an engineer at a small aerospace firm, who was recruited 
from a list of recently hired engineers provided by one of the directors. The company is an 
engineering firm focused on providing innovative and entrepreneurial aerospace products to 
commercial, civil, and military customers. 

One of the interviewees is female and six are male. The interviews were conducted between the 
months of November 2006 and May 2007, and ranged in length from 33 minutes to 74 minutes. 
The interview guide used for these interviews is included in Appendix 3-F. 

Issues explored in these interviews include: 
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• The new engineers’ retro speetive views of formal edueation 

• The degree to which their work permeated home life 

• The extent to which the new engineers felt that engineering should be in the service of 
others 

• How these engineers learned new technical skills at work 

• How they learned new social skills at work 

• Their satisfaction in practicing the craft of engineering 


A listing of papers and reports from the Workplace Cohort research can be found at 
http://www.engr.washington.edu/caee/ . 
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Appendix 1-A 

APS File Naming Protocol 


Data File Naming 

Names of data files within the APS database as proposed here have 6 standard components (plus 
one optional component) organized in the following order: 

1 . StudentID: coded per CAEEID 

2. MethodType: instrument or method 

3. EventID: event_sequence or event_date, or combination 

4. ItemID: item_type, item_number and revision_number 

5. ResearcherlD: name initials 

6. (optional) Pseudonym: reference subject’s “name” 

7. EilenameExtension: document_type 

To improve readability and facilitate accurate computer-based parsing, filename components are 
separated by hyphens. By design, the resultant filename will uniquely identify the context of 
each data file in the APS database. This filenaming convention would result in filenames that 
look l ik e 


StudentlD - MethodTvpe - EventlD - ItemlD - ResearcherlD - Pseudonym . EExt 


This document is a work in progress and may be revised to reflect new needs and functions of 
the APS database. It is important that this proposed filenaming protocol not only meet the 
immediate needs related to the current study of the Longitudinal Cohort (Cohort 1), but can also 
be carried forward to integrate data for future APS cohort studies. 

StudentID 

In the APS database, we must avoid identifying our study’s student participants by real names or 
other recognizable real-world information, in association with collected data. To abstract a 
participant’s identity, we have developed a coding scheme. This CAEE Student ID code 
uniquely identifies each student participant and can be broken down into 4 parts. It looks 
something like: "TPubOlEOOOOS". The first part, "TPub", is the school's official acronym (i.e.. 
Technical Public Institution). The second part, "01", is the cohort id. The third part, "E", refers to 
the gender, female. The fourth and final part, "00003", is a sequentially generated number 
identifier of the student at the indicated school. 

Eor cohort- 1, we have TPub, UPri, SPri, and LPub as the 4 possible school acronyms. When 
expanding the study in cohort-3, and cohort-4, most U.S. schools have unique acronyms. If we 
should encounter two or more schools with identical acronyms, we can append a lower case 
letter (a, b, c...) in sequence to differentiate these schools. Internal to the APS database, there is a 
table that connects this acronym (known as UnivID) to the full name of the school and its related 
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descriptive data (e.g., university's full name, semester/quarter system, etc.). 

With this StudentID code displayed in the filename, a researcher can quickly tell at a glance that 
the data file is associated with a specific student who attends a given school, is of a given gender, 
and participates as a member of a specific cohort. 

If student data were aggregated into container documents by school, the containers’ filenames 
would only include the school acronym and the cohort id (e.g., “TPubOl”). No gender and 
student identifying sequence number would be included. 

MethodType 

The MethodType component in the filename identifies the data instrument or method being used 
to collect the data contained in the file. This portion of the filename is typically 3-4 letters long. 
For our research as planned, the following MethodTypes would be used: 

SURV survey data 

INTS structured interview data, 

INTE ethnographic interview data 
INTX exit interview data 

INSP problem scoping exercise data within structured/ethno interview session 
ETD? engineering thinking and doing data 
ACTX academic transcript data 
ETH? ethnography data 

EventID 

The EventID is used to identify the particular instance of the data collection event. The EventID, 
taken together with MethodType, allow us to refer to a specific data set in a sequence, such as 
Survey 2, or Structured Interview I. We will use numeric digits such as { 1, 2, 3} to specify the 
EventID. 

Ethnographers will typically contribute new files on a regular basis throughout the year. It may 
be more appropriate for such research methods to use date in lieu of a sequence number to 
represent the data gathering event. When applied, the date would be formatted as “YYMMDD”, 
so as to facilitate chronological sorting. This date information should not substitute for the 
inclusion of more detailed date information inside the file document itself. 

ItemID 

The ItemID is used to identify one or more data items collected together within the context of a 
single data collection event. It is composed of three parts, in order: 
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1. DataType, 

2. ItemNumber, and 

3. RevisionNumber. 


For example, in the course of structured interview #1, we may produce one audio recording file, 
one interview notes document, and 2 PDF scan files. In this scenario, we would have 4 files with 

MethodType = INTS 
EventID=l 

and the following distinguishing ItemIDs 

ItemID=Al_l (Audio File) 

ItemID=Nl_l (Notes File) 

ItemID=Sl_l (Scan File #1) 

ItemID=S2_l (Scan File #2) 


If after review, the notes file with ItemID=Nl_lwas revised and resubmitted to the database, the 
revised file would take on ItemID=Nl_2. 

At the current time, the following DataType codes are proposed: 

A audio recordings 
V video recordings 
N notes (field notes, interview notes) 

T text transcriptions 
R transcript revision requests 
E MS Excel data 
X tab delimited columnar data 
C comma delimited data 
H Survey rendered from HTME 
S SPSS data analysis 
P digital photos 
Z scanned paper documents 


ResearcherlD 

The ResearcherlD identifies the researcher who is primarily responsible for collecting the data in 
the file. The researcher’s initials (in all capital letters) will be used. If we should encounter a 
situation in which a new researcher has initials identical to an existing researcher, we would 
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append a number to the new researcher’s initials for the ResearcherlD. For example, if we have a 
researcher Gwendelyn Talbot and we add a new researcher Greg Taylor, Gwendelyn Talbot 
would have ResearcherlD “GT” and Greg Taylor would be assigned “GTl”. 

Pseudonym 

Ethnographers will typically contribute new files on a regular basis throughout the year. In such 
cases, the CAEE student ID may be hard to write and refer to in discussion. The student 
pseudonym, appended to the root part of the document name and also recorded in the APS 
database, would be the identifier that ethnography researchers would use to refer to the student 
subject. 

Student pseudonyms would be created by the research teams at each university, and may be 
something like "Eegol". Each pseudonym is unique within the subject group at each school; it is 
not permitted to have another "Eegol" within that school’s subject group. However, it is entirely 
acceptable to have "Eegol" at another school. This way, researchers have full autonomy and 
flexibility to choose pseudonyms without fear of conflicts with those chosen by researchers at 
other universities. 

Pseudonym, this filename component, is optional and is not likely used outside of ethnography 
documents. 

FilenameExtension 

We use familiar filename extensions to identify their respective applications and data types. The 
following example filename extensions would be used with associated applications: 

.rtf Microsoft Word, Mac TextEdit 

.doc Microsoft Word 

.dot Microsoft Word (Template) 

.xls Microsoft Excel (spreadsheet) 

.CSV Microsoft Excel (comma separated values) 

.sav SPSS data file 

.sps SPSS variable definitions/label file 

.dss Olympus Audio Recorder/Player 

.pdf Adobe Acrobat Reader, Mac Preview 

.txt Microsoft Notepad, Mac TextEdit 

.zip WinZip, Windows Explorer (XP), Stuffit Expander 

.jpg Internet Explorer, Netscape, Mozilla, Safari 

.tif (tagged image format) 
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Appendix 1-B 

APS Data Access Guidelines 


Researchers will be able to access data needed for their research through the APS database. 
However, it is important that we, as APS researchers-individually and as a community, handle 
APS data responsibly. 

This data access guideline is intended to facilitate adherence to IRB research procedures and 
obligations. The goal is to minimize the likelihood for accidental data sharing with those for 
whom such data access may violate conflicts of interests, IRB approved research protocols, 
and privacy laws. Toward this end, the procedures presented here are designed to help 
researchers identify the data type, recognize data handling risks, and acknowledge 
responsibilities. 


1 Data Types 

Some APS data sets are more sensitive than others. To help us differentiate, we can classify our 
APS data sets as 3 different types: raw, anonymized, or aggregated. 

Raw data describes individual student attributes or performance in a way that allows a 
researcher to identify individual students — by voice, by appearance (picture), by name, 
by a standardized identifier used in the study (CAEE_ID), or any other similar means. 
Thus, data files that include the student’s CAEE_ID in its filename must also be 
considered raw. Raw data is highly sensitive. It must be carefully secured at all times, 
and should not be shared with anyone unnecessarily. Typically, only those researchers 
involved in the specific data collection process would need access to the respective raw 
data set. 

Anonymized data can be prepared from raw data by either stripping all student 
identifiers, or replacing identifiers with newly created codes which would be unknown to 
any other researcher. Using a private set of pseudonyms is one way of anonymizing 
student data. While real student identities cannot be traced, data of individual students 
can be collected and tracked. Thus, anonymized data is moderately sensitive. 

Aggregated data is prepared by grouping, processing, categorizing data of multiple 
students statistically, numerically, or descriptively. In such case, no individual students 
are distinguishable or identifiable. With any reasonably large grouping, aggregated data 
is descriptive of its constituent population. No individual student information can be 
extracted. Thus, aggregated data is the least sensitive. 


2 Risks and Responsibilities 

Prerequisite: all researchers that have (or are expected to have) contact with the study’s subjects 
and/or their data must first complete human subjects research (IRB) training. 
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Researchers are individually responsible for continuously reviewing their own risks and 
concerns, when accessing and handling APS data. To avoid accidentally exposing information 
to an inappropriate recipient, individual researchers should make their own sets of recognized 
risks known to others. For example, if researcher_X should not receive information_Y because 
of a particular conflict of interest risk, researcher_Z who has information_Y should be alerted. 
In this way, APS researchers share responsibility to alert one another. 


3 Research Data Access Requests by Category of Researcher; 

A data sharing arrangement for the duration of the active grant was developed near the start of 
the project and is described in general for the three researcher categories below. Any deviation 
from these data sharing arrangements could occur with permission from the project leads and 
subsequent final approval from APS lead, Sheri Sheppard. Data sharing following the end of the 
grant period is outlined in section 5 Data Academic Pathways Study Data Sharing Agreement 
below. 


Category 1. APS School Researcher: School PI approved 

Each school PI would enable access to APS researchers at their own campus to their school's 
data. It would be up to this PI to decide what types (e.g., method, and raw/anonymized, 
aggregated) each of his/her researchers should have access to, and to ensure that each researcher 
has completed human subjects training and has been apprised in the APS risks and responsibility 
statement. New campus researchers would be added by the PI, as needed. 

• LONGTERM APPROACH: Pis would enable access for each researcher via a web form. This 
form would automatically enable access and would post the status of the researcher's access 
privileges on the database (so that everyone knows of everyone else's access). 

• SHORTTERM APPROACH: until the web form is up and active. Pis should send a list of their 
campus researchers to the database manager. 

Category 2. APS Method Researcher: Method lead compiled, Cognizant APS Lead 
Reviewed. 

Each method lead (ethnography, surveys, structured interviews, performance task, academic 
transcripts, and Cohort 2) would provide cognizant APS Lead PI with a list of researchers who 
should have access to data associated with a particular method across ALL schools for the 
purpose of cross-campus analysis on a single method. In general these will be a list of 
researchers who have responsibility for single-method analysis at all institutions and who are 
working closely with the method lead. Cognizant APS Lead PI would then enable access to APS 
researchers to that data type. It would be up to this PI to decide what types (e.g., method, and 
raw/anonymized, aggregated) each of his/her researchers should have access to, and to ensure 
that each researcher has completed human subjects training and has been apprised in the APS 
risks and responsibility statement. New cross-campus researchers would be added by the 
cognizant APS Lead PI, as needed. 

• LONGTERM APPROACH: Cognizant APS Lead Pis would enable access for each researcher 
via a web form. This form would automatically enable access and would post the status 
researcher's access privileges on the database (so that everyone knows of everyone else's access). 
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• SHORTTERM APPROACH: until the web form is up and aetive, Pis should send a list of their 
campus researchers to the database manager. 

Category 3. APS Researcher. 

APS researcher who wants data across all four campuses for the purposes of multi-method and/or 
multi- institution APS analysis, submits a web form to the APS Lead team. The form would ask 
that the researcher briefly describe the research question that they are pursuing, how it relates to 
core APS research questions, and what type of analysis they are undertaking. This form is 
reviewed on one of the monthly/bi-monthly APS Lead calls. If urgent, the review could happen 
via email (championed by the campus APS lead). Each school has the right to veto access to 
his/her school's data. Approved Category 3 web form is then forwarded to database manager 
who enables access (if there is way to automate this enabling, we will). 


4 Recommended Data Handling Guidelines 

1. Whenever possible, data should be transferred via the APS database. 

2. Unencrypted data should NOT be transferred via unsecured mail or email. 

Sending data via secure mail or email guarantees that only the intended recipient is able 
to receive and read the contents of the delivered package. Lor email, this typically means 
applying digital key encryption and digital signatures on the data package before sending. 
Lor mail, this may mean physically locking the package in such a way so that an 
unintended recipient would be prevented from opening it. 

3. All computers and media containing APS data must be carefully secured and protected 
from compromise by hackers/virus/spyware/Trojan horse programs/network attacks. 

4. Here are a few things to review: 

• is your computer secured from theft? 

• is your computer used or accessible by other people? 

• is your computer data files accessible by other users? 

• is your computer ever connected to a network? 

• is your computer protected by a hardware or software network firewall? 

• does your computer run a virus checker with daily or weekly updates? 

• is your computer running a “malware” checker with daily or weekly updates? 

• are your data files being backed up? 

• are your backups protected from theft? 

• are your backups accessible by other people? 

5. In general, researchers should avoid sharing any raw APS data, student contact 
information and even names of students’ participation in the study. Lor example, in cases 
with faculty researchers, knowledge of study participant’s identities may pose a conflict 
of interest, or a potential conflict of interest in future years of the study. These represent 
undesirable outcomes. Therefore, assume APS raw data, student participation or contact 
information is not sharable. Such information should not be shared publicly, or shared in 
an insecure forum. 
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5 Academic Pathways Study Data Sharing Agreement 


This agreement establishes the policy to address data sharing among the Academic Pathways 
Study leads during the concluding phase of the grant (beginning with the initial distribution of 
CD#1 in early 2009) and continuing after the grant ends (September 30, 2009). 

1) The APS Core Data set consists of a 4-box CD set: 

CD#1 (Cohort 1) (late 2008/early 2009) 

CD#2 (Cohort 2) (3rd quarter 2009) 

CD#3 (Cohort 3) (first quarter 2009) 

CD#4 (Cohort 4) (2nd quarter 2009) 

Each APS CO-PI from the four primary schools receives a 4-box CD set of Core APS data. In 
addition, the APS co-PI from the remaining school receives CD#2 and CD#4. 

2) When a new topic/question is being explored with the Core APS data that involves cross- 
school data, email notification to the APS co-PI team is required. If the topic/question is to be 
explored using only data from the cognizant APS Lead's school, notification is recommended. 

3) Any publications coming out of said research to have an APS co-PI as a co-author. In 
addition, any and all references to schools should be via the established pseudonyms. 

4) Any research using this core data needs to be under the supervision of one of the APS co-PIs. 
The supervising APS co-PI is responsible for ensuring that proper IRB approval and training 
have been obtained for all associated researchers. 

5) Secondary data (e.g.. Cohort T, NSSE) to be held by cognizant APS co-PI, and requests to use 
that data to be sent to that APS co-PI. 

6) Any publications coming out of said research should include a statement acknowledging NSE 
Grant No. ESI-0227558. 


Supplement A: 

Research Data Access (RDA) Request Form (Category 3) 


W 


hat: 


Data Set ID: 

(see Appendix B) 
(data_type + school(s) + 
collection_event_date) 
[highlight in table] 

[ Survey I Structured_lnterview I Performance _Task I Exit_Interview I 
Ethnographic_Interview I Academic _Record I [Cohort 2 
Interviews...] 

[ school identifier^ 

[ Event _ID ] 

NOTE: Data Set ID would have format similar to APS data filenaming convention. 

Data Classification: 

[ Raw I Anonymized I Aggregated] 


Whm 

I Researcher Name: 
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Researcher Institution: 


Researcher Role(s) (relative to data set): 

[ APS Data Owner I Campus Data Owner I APS_ 
Researcher I CAEE_Researcher ] 

Affiliated CAEE PI (if different than 
researcher) 


Has researcher completed IRB training? 

(circle one) YES NO 


D uration; 

I Date when data set access will end: 


B enefits: 

Briefly describe analysis plan for use of data set: 


Briefly explain expected outcome of analysis, why data access is beneficial or required: 


Risks; 

Identify potential risks associated with researcher’s data access and describe how these are (or will be 
addressed): 


Identify persons who should not see data set(s), and why, if any: 


E xpected Outcomes; 

What type of analysis results can or will be made available to others? 

Analysis results data file format (i.e. software applications used)? 
Data type of results: raw, anonymized, or aggregated? 
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Appendix 1-C 

APS Persistence Definitions (Longitudinai Cohort /Cohort 1) 


Date: 17 September 2007 


Academic Persistence and Non-persistence 
Persistence = Declared an engineering major 

Non-persistence = Declared a non-engineering major after having intent to study 
engineering at admission to university 

Guidelines for determining persistence from APS Longitudinal Cohort (Cohort 1) data 

(1) Majors are determined to be engineering or non-engineering on a school-by- 
school basis. If the subject chooses a major within the school of engineering at 
his/her institution after having the intent to study engineering at admission, the 
student is considered a "persister". (Therefore some majors, such as computer 
science, may be considered an engineering major at some institutions and not 
others). 

(2) A students persistence status is determined when s/he formally declares her/his 
major. (The major must be reflected in the academic record, i.e., stating an intent 
to declare a specific major in an APS interview will not change their persistence 
status.) 

(3) For subjects who are not listed as persisters or non-persisters, we do not have 
enough information to classify them. (For example, the subject may have 
dropped out of the study and researchers are unable to track him/her down - or the 
subject has not yet formally declared a major.) 

(4) Subjects with double (or more) majors of which (at least) one is an engineering 
major, are classified as persisters. 

(5) Persistence status should be considered subject to change: 

a. Persistence status is periodically revisited (-twice a year). (I reanalyze 
previous classifications to update persistence status, for example, there are 
still subjects declaring non-engineering majors - and we've had one 
subject declare an engineering major after having declared a non- 
engineering major). 

b. Persistence status will be finalized once transcript analyses are completed. 
Note: In the Longitudinal Cohort, we have one exception who is a CS major (CS is not 
considered engineering at one APS partner institution) - this student was recruited as a 
CS student and has remained a CS student. He was initially considered a persister, but as 
of August 2007 will be excluded from persistence analysis. 


Migrator 

Student who initially did not intend to study engineering at admission to university, but 
later declared an engineering major. Please see: K.M. Donaldson, S.D. Sheppard (2007), Exploring 
the Not-So-Talked-About Undergraduate Pathway: Migrating Into Engineering. Paper presented at the 
2007 International Conference on Research in Engineering Education, Honolulu, Hawaii, June 23-24, 2007. 
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Dropped out 

Longitudinal Cohort subject who left the university they were attending during the APS 
study 

Exit 

When a Longitudinal Cohort subject declares a non-engineering major (same as non- 
persister). Not all “exited” students had exit interviews. (For people who are no longer a 
part of the study, please say they “left the study.”) 

Exit Interview 

A special semi-structured interview given to Eongitudinal Cohort subjects who formally 
declared a non-engineering major (or exited engineering). 

Full data 

Full data refers to subject data sets that are complete for their contact grouping, e.g., the 
low contact group has a full data set for 24 subjects at Orchard University (24 Orchard 
subjects participated in all of the surveys and APS has all of their transcripts). 

Incomplete data 

Incomplete data refers to a subject data set that is not complete, e.g., a Eongitudinal 
Cohort subject took five (out of a total six) surveys. 

Feet the study 

Used to describe Eongitudinal Cohort subjects who left APS before the study ended - 
therefore they have an incomplete data set (e.g., they took three surveys and then no 
longer took part in the study). We do not characterize these students. 

Proeessional Persistence 

Subject intends to work in engineering for three years following graduation with their 
undergraduate degree. 

Returned 

Returned refers to a Eongitudinal Cohort subject who exited engineering and then 
returned to engineering, i.e., declared a non-engineering major and then changed majors 
to an engineering major. 

Unknown 


Refers to Longitudinal Cohort subjects whose persistence status(es) are unknown based 
on conversations with researchers at the associated Longitudinal Cohort institutions. 
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Appendix 1-D 

APS Representative Exampies of Informed Consent Forms 
Technicai Pubiic Institution (administered by Urban Private University) 


TPub Year 1 D-2 

Urban Private University 

UPri Year 2 D-6 

Suburban Private University 

SPri 9/26/03 to 5/29/04 D-10 

SPri 4/28/06 to 4/28/07 D-12 

Large Pubiic University 

LPub General Consent Form, APS D-14 

LPub Consent Form, APS, Ethnographic Group D-17 

LPub Consent Form, APS, Observed Group (rev 05/08) D-24 


Note: Informed Consent forms for Urban Private University and Technical Public Institution 
used the same text with the exception of the school name and other school-specific details. 
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TECHNICAL PUBLIC INSTITUTION^ PARTICIPANT CONSENT FOR 
INVESTIGATIVE PROCEDURES 
URBAN PRIVATE UNIVERSITY 


The following instruments, interviews, or procedures are needed for the research project entitled, 
“Scholarship on Learning Engineering Project.” 

Instruments, Interviews, or Procedures to be Employed 

A. Surveys will be administered to participants in the observational group to ascertain 
information about students’ perceptions, experiences and motivations as an 
undergraduate engineering student. 

B. Individual interviews will be administered to participants in the observational group to 
obtain a deeper understanding of students’ perceptions, expectations, motivations, 
practices, and experiences in engineering. 

C. Ethnographies will be conducted on some participants in the observational group in order 
to obtain a general impression of undergraduate engineering student life. 

D. Student academic and admission records will be obtained for participants in the 
observational group. 

E. Student academic and admission records will be obtained for participants in the control 
group. 

Explanation to Participants 

You are invited to participate in a national research study funded by the National Science 
Eoundation (NSE). The purpose of this study is to gain an in-depth understanding of how 
engineering students explore and utilize the educational opportunities available to them in 
college, plan and carry out their curriculums (successfully or unsuccessfully), and navigate the 
process of becoming engineers. The study will take place over a period of three years, between 
Eall 2003 and Spring 2006. You will not need to do any more or less work in your classes 
because you have agreed to participate in this study. Nor will you be asked to perform 
differently than you would if you did not participate in this study. As discussed in the 
information meeting, a combination of the following methods will be used in the study to capture 
different aspects of your experience as an engineering student. These methods and procedures 
are as follows: 

You will be asked to: 

1) Complete two 2-hour Internet-based questionnaires in which you will be requested to 
provide some information about your experiences and motivations as an undergraduate 
engineering student. 

2) Participate in a 2-hour interview per year in which a trained observer will ask of you 
standardized questions regarding your experiences and motivations as an undergraduate 


* Informed consent for Technical Public Institution (TPub) was administered by Urban Private University because 
TPub does not have its own IRB office. 
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engineering student. These interviews will be audio taped. The audiotapes will be 
destroyed five years after the study is completed. 

3) Participate in a series of non-intrusive observations in which a trained observer will 
accompany you for no more than 2 hours a week in various academic (e.g., lectures, 
study groups) and as well as non- academic settings (e.g., dormitory, extracurricular 
activity settings) in order to obtain a general impression of undergraduate engineering 
student life. Observations will be recorded in field notes and through the occasional use 
of digital photography, audiotape, and videotape. Photos and videos taken of the activity 
being conducted will feature only the individual’s body and not his or her face. The 
digital photography, audiotapes, and videotapes will be destroyed five years after the 
study is completed. 

4) Authorize access to your academic information maintained by the university registrar, 
such as SAT scores, high school GPA, college transcripts, gender, and ethnicity. This 
information will be kept in strictest confidence and will only be used for research 
purposes. 

There is some risk associated with all research. However, the risks to you are no greater than 
would be expected from participation in any educational research activity. The potential risks are 
the inappropriate use of the information that is collected and uneasiness from participation in the 
interview and observation processes. To protect against these potential risks, the research 
personnel have signed a confidentiality agreement that limits access and use of all study data. 
Moreover, the trained interviewers and observers will take extra precaution to ensure that you are 
as comfortable as possible. 

There are a number of benefits for participating in this study. The study seeks to identify and 
characterize the pathways for making the choice of becoming an engineer. You will have an 
opportunity to contribute your experiences in this process. This understanding will enable the 
engineering community to pinpoint and clearly define areas in engineering education, where 
additional work is needed for improvement. 

You will be compensated in cash for participating in this study. Legally, you are only eligible for 
compensation if you are a U.S. citizen, a legal resident alien (i.e., possess a "green" card), or 
have a work eligible visa sponsored by the paying institution. The amount of compensation will 
depend on your level of participation. There will be two groups of participants: an observational 
group and a control group. Observational group participants will be compensated $175.00 per 
year for participating in a combination of survey, interview, and observation methods. 
Observational group participants will also be given a scientific calculator during the spring 
semester of the first year. Control group participants will be compensated $25.00 per year for 
authorizing access to student admission and registrar records. Annual payments will be made at 
the end of spring semester upon continuous participation for that school year. There will be an 
equal number of participants in each group. However, if you chose to participate, it is not 
possible to know which group you will be assigned to before the researchers have obtained 
consent from all potential participants. 

All information we gather on this project will remain strictly anonymous and will not identify 
you in any way. The interview sessions will be audio taped, and the ethnographic sessions will 
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be audio taped, photographed, and videotaped so that we can review them later. However, all 
identifying information will be removed and replaced by code numbers. Participant codes will 
be kept in a logbook, which will be stored in a locked file cabinet separate from the actual data 
information. We will keep all materials in a secure place in our university office. Information 
will be reported based on group profiles and averages, and not on individual responses. At no 
time will information be reported or identified with the participant’s name, guaranteeing that the 
privacy of all participants will be protected. All information that we gather will be destroyed by 
December 2012. 

Your participation in this project is completely voluntary, so you can stop taking part at any time 
without jeopardizing your relationship with the university institution, engineering department, 
project researchers, or related staff. 

Participant’s Statement of Understanding 

I may be asked to participate in several activities throughout the school year, such as surveys, 
interviews, and observations, as well as authorize access to my academic admission and registrar 
records for research purposes only. 

Some of the tests or procedures described may be novel or experimental, but they do not 
involve any risk(s) other than those described above. In the event of physical or other 
injury resulting from the research tests or procedures, emei^ency medical treatment will 
be provided, but financial compensation will not be available. 

With this knowledge and the above description of the project, I voluntarily agree to take part, 
accepting the risk(s) of my participation in this project. I further understand that all reasonable 
precautions have and will be taken to reduce the risks(s) and to provide for my care. 

I am aware that I am free to withdraw this consent and discontinue participation in this project at 
any time without affecting either my ability to receive on-going care or my relationship with 
Technical Public Institution or Urban Private University. 

The Principal Investigator or Project Researcher, at Technical Public Institution may be 
contacted at [phone #] in the event that I have any questions regarding my participation in this 
project. The Urban Private University Institutional Review Board will have access to the records 
of this project. If I have questions regarding my rights as a study participant that I would like to 
discuss with someone other than the investigators on this project, I am free to contact the Office 
of Executive Secretary, Urban Private University Institutional Review Board at [phone #]. 

The study described above has been explained to me. My willingness to participate is 
indicated below. (Please check appropriate box.) 

n I do not wish to participate in this activity. In order for us to ensure that you will 

not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 


1D-4 


APS Research Processes and Procedures 

January 2009 



CH I voluntarily consent to participate in this activity. I have had an opportunity to ask 
questions. I give permission for the study team to use my responses in research. 
(Please proceed to the following sections). 

I give consent to be interviewed during this study. (Please initial): Yes No 

I give consent for my interview to be audio taped. (Please initial): Yes No 

I give consent for the responses I provide during the interviews and surveys to be used for 
research purposes in this study. (Please initial): Yes No 

I give consent to be observed during this study. (Please initial): Yes No 

I give consent for my observation to be audio taped, photographed, and videotaped. I understand 

that my face will not be shown in any photograph or videotape. (Please initial): Yes 

No 

I give consent for my photographed, and videotaped observation to be used during presentations 

concerning the research, such as at a professional conference. (Please initial): Yes 

No 

I give consent for my academic records to be used for research purposes in this study. 

(Please initial): Yes No 

I have read the above description of the research project. A member of the research team has 
answered all of my questions to my satisfaction. I agree to participate in the above-referenced 
project. I acknowledge that I have received a personal copy of this consent form. 

Participant’s Signature Date 

I, the undersigned, have defined and fiilly explained the tests and procedures involved in this study 
to the above participant. 

Researcher’s Signature Date 

Witness’ Signature Date 
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URBAN PRIVATE UNIVERSITY PARTICIPANT CONSENT FOR 
INVESTIGATIVE PROCEDURES 
URBAN PRIVATE UNIVERSITY 


The following instruments, interviews, or procedures are needed for the research project entitled, 
“Scholarship on Learning Engineering Project.” 


Instruments, Interviews, or Procedures to be Employed 


A. Surveys will be administered to participants in the observational group to ascertain 
information about students’ perceptions, experiences and motivations as an 
undergraduate engineering student. 

B. Individual interviews will be administered to participants in the observational group to 
obtain a deeper understanding of students’ perceptions, expectations, motivations, 
practices, and experiences in engineering. 

C. Ethnographies will be conducted on some participants in the observational group in order 
to obtain a general impression of undergraduate engineering student life. 

D. Student academic and admission records will be obtained for participants in the 
observational group. 

E. Student academic and admission records will be obtained for participants in the control 
group. 


Explanation to Participants 

You are invited to participate in a national research study funded by the National Science 
Foundation (NSE). The purpose of this study is to gain an in-depth understanding of how 
engineering students explore and utilize the educational opportunities available to them in 
college, plan and carry out their curriculums (successfully or unsuccessfully), and navigate the 
process of becoming engineers. The study will take place over a period of three years, between 
Fall 2003 and Spring 2006. You will not need to do any more or less work in your classes 
because you have agreed to participate in this study. Nor will you be asked to perform 
differently than you would if you did not participate in this study. As discussed in the 
information meeting, a combination of the following methods will be used in the study to capture 
different aspects of your experience as an engineering student. These methods and procedures 
are as follows: 


You will be asked to: 

1) Complete two 2-hour Internet-based questionnaires in which you will be requested to 
provide some information about your experiences and motivations as an undergraduate 
engineering student. 
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2) Participate in a 2-hour interview per year in which a trained observer will ask of you 
standardized questions regarding your experiences and motivations as an undergraduate 
engineering student. These interviews will be audio taped. 

3) Participate in a series of non-intrusive observations in which a trained observer will 
accompany you for no more than 2 hours a week in various academic (e.g., lectures, 
study groups) and as well as non- academic settings (e.g., dormitory, extracurricular 
activity settings) in order to obtain a general impression of undergraduate engineering 
student life. Observations will be recorded in field notes and through the occasional use 
of digital photography, audiotape, and videotape. Photos and videos taken of the activity 
being conducted will feature only the individual’s body and not his or her face. 

4) Authorize access to your academic information maintained by the university registrar, 
such as SAT scores, high school GPA, college transcripts, gender, and ethnicity. This 
information will be kept in strictest confidence and will only be used for research 
purposes. 

There is some risk associated with all research. However, the risks to you are no greater than 
would be expected from participation in any educational research activity. The potential risks are 
the inappropriate use of the information that is collected and uneasiness from participation in the 
interview and observation processes. To protect against these potential risks, the research 
personnel have signed a confidentiality agreement that limits access and use of all study data. 
Moreover, the trained interviewers and observers will take extra precaution to ensure that you are 
as comfortable as possible. 

There are a number of benefits for participating in this study. The study seeks to identify and 
characterize the pathways for making the choice of becoming an engineer. You will have an 
opportunity to contribute your experiences in this process. This understanding will enable the 
engineering community to pinpoint and clearly define areas in engineering education, where 
additional work is needed for improvement. 

You will be compensated in check for participating in this study. Legally, you are only eligible 
for compensation if you are a U.S. citizen, a legal resident alien (i.e., possess a "green" card), or 
have a work eligible visa sponsored by the paying institution. There will be one group of 
participants: an observational group. Observational group participants will be compensated 
$175.00 per year for participating in a combination of survey, interview, observation methods, 
and review of academic information. Annual payments will be made at the end of spring 
semester upon continuous participation in surveys and interviews for that school year. 

All information we gather on this project will remain strictly anonymous and will not identify 
you in any way. The interview sessions will be audio taped, and the ethnographic sessions will 
be audio taped, photographed, and videotaped so that we can review them later. However, all 
identifying information will be removed and replaced by code numbers. Participant codes will 
be kept in a logbook, which will be stored in a locked file cabinet separate from the actual data 
information. We will keep all materials in a secure place in our university office. (All encryption 
documents relating your randomly generated ID codes to your identity will be destroyed by 
December 2012. Data files will be archived until they are no longer needed.) At no time will 
information be reported or identified with the participant’s name, guaranteeing that the privacy 
of all participants will be protected. 
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Your participation in this project is completely voluntary, so you can stop taking part at any time 
without jeopardizing your relationship with the university institution, engineering department, 
project researchers, or related staff. 

Participant’s Statement of Understanding 

I may be asked to participate in several activities throughout the school year, such as surveys, 
interviews, and observations, as well as authorize access to my academic admission and registrar 
records for research purposes only. 

Some of the tests or procedures described may be novel or experimental, but they do not involve 
any risk(s) other than those described above. In the event of physical or other injury resulting from 
the research tests or procedures, emergency medical treatment will be provided, but financial 
compensation will not be available. 

With this knowledge and the above description of the project, I voluntarily agree to take part, 
accepting the risk(s) of my participation in this project. I further understand that all reasonable 
precautions have and will be taken to reduce the risks(s) and to provide for my care. 

I am aware that I am free to withdraw this consent and discontinue participation in this project at 
any time without affecting either my ability to receive on-going care or my relationship with Urban 
Private University. 

The Principal Investigator or Project Researcher, may be contacted at [phone #] in the event that I 
have any questions regarding my participation in this project. The Urban Private University 
Institutional Review Board will have access to the records of this project. If I have questions 
regarding my rights as a study participant that I would l ik e to discuss with someone other than the 
investigators on this project, I am free to contact the Office of Executive Secretary, Urban Private 
University Institutional Review Board at [phone #] . 

The study described above has been explained to me. My willingness to participate is 
indicated below. (Please check appropriate box.) 

n I do not wish to participate in this activity. In order for us to ensure that you will 

not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 

n I voluntarily consent to participate in this activity. I have had an opportunity to ask 
questions. I give permission for the study team to use my responses in research. 
(Please proceed to the following sections). 

I give consent to be interviewed during this study. (Please initial): Yes No 

I give consent for my interview to be audio taped. (Please initial): Yes No 

I give consent for the responses I provide during the interviews and surveys to be used for 

research purposes in this study. (Please initial): Yes No 

I give consent to be observed during this study. (Please initial): Yes No 

I give consent for my observation to be audio taped, photographed, and videotaped. I understand 
that my face will not be shown in any photograph or videotape. (Please initial): Yes No 
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I give consent for my photographed, and videotaped observation to be used during presentations 

concerning the research, such as at a professional conference. (Please initial): Yes No 

I give consent for my academic records to be used for research purposes in this study. 

(Please initial): Yes No 


I have read the above description of the research project. A member of the research team has 
answered all of my questions to my satisfaction. I agree to participate in the above-referenced 
project. I acknowledge that I have received a personal copy of this consent form. 


Participant’s Name (Print) 

Participant’s Signature Date 


I, the undersigned, have defined and fully explained the tests and procedures involved in this study 
to the above participant. 

Researcher’s Signature Date 

Witness’ Signature Date 
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NSF Center for the Advancement of Engineering Education 
CONSENT EORM: Eongitudinal Study 


EOR QUESTIONS ABOUT THE STUDY, CONTACT: Project Researcher at Suburban Private 
University [contact information]. 

DESCRIPTION: 

You are invited to participate in a national research study funded the National Science Foundation (NSF), 
which aims to gain an in-depth understanding of how engineering students explore the educational 
opportunities available to them in college, plan and carry out their curriculums (successfully or 
unsuccessfully), and become engineers. Tbe study will take place between Fall 2003 and Spring 2006. 
You will not need to do any more or less work in your classes because you have agreed to participate in 
this study, nor will you be asked to perform differently than you would if you did not participate. As 
discussed in the information meeting, a combination of tbe following methods will be used in tbe study to 
capture different aspects of your experience as a student who is planning to major in engineering: 

1) One 2 hour interview per year, in which a trained observer will ask of you standardized questions 
regarding your experiences and motivations as an undergraduate pre-engineering student. The interviews 
will be audiotaped. 

2) Two 2 hour internet -based questionnaires, where you will be requested to provide some 
information about your experiences and motivations as an undergraduate pre-engineering student. 

3) A series of non-intrusive observations in which a trained researcher will accompany you for no 
more than 2 hours a week in various academic (e.g., lectures, study groups) and as well as non-academic 
settings (e.g., dormitory, extracurricular activity settings) in order to obtain a general impression of 
undergraduate pre-engineering student life. Tbe researcher will take notes and occasional photographs in 
order to document his/her observations, and might ask informal questions. 

4) Access to your academic information maintained by the university registrar such as your SAT 
scores, high school GPA, college transcripts, gender, and ethnicity. This information will be kept in 
strictest confidence and will only be used for research purposes. 

All information collected will be kept confidential, and accessed only by authorized researchers who have 
signed a confidentiality agreement. To better preserve your privacy, a random identification code will be 
used in place of your name during the study. This is the same code that will be used in the interviews and 
questionnaires you have been asked to participate in. 

RISKS : 

The potential risk is the inappropriate use of the information that is collected. To guard against this, the 
research personnel have signed a confidentiality agreement that limits access and use of data. (All 
encryption documents relating your randomly generated ID codes to your identity will be destroyed by 
December 2012. Data files will be archived until they are no longer needed.) Also, some people feel self- 
conscious when they are observed for the purposes of a study. 

BENEFITS: 

The study seeks to identify and characterize the pathways for making the choice of becoming an engineer. 
This understanding will enable the engineering community to pinpoint and clearly define areas in 
engineering education, where additional work is needed for improvement. 

PAYMENTS: 

You will be compensated in cash. Legally, you can be paid only if you are a US citizen, a legal resident 
alien (i.e, possess a "green" card), or have a work eligible visa sponsored by the paying institution. The 
amount of compensation will depend on your level of participation. There will be two groups of 
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participants: a detailed study group, and a control group. Detailed study participants will be observed 
through a combination of any of the four methods described in the “description” section of this form, and 
compensated $175 per year. Detailed group participants will also be given a scientific calculator 
during the winter quarter of the first year. Control study participants will be observed through the 
fourth method only, and compensated $25 per year. Annual payments will be made at the end of spring 
quarter upon continuous participation for that school year. There will be an equal number of participants 
in each group. However, if you chose to participate, it is not possible to know which group you will be in 
before the researchers obtain consent from all interested students. 

SUBJECT’S RIGHTS: 

If you have read this form and have decided to participate in this project, please understand that your 
participation is voluntary and you have the right to withdraw your consent or discontinue participation at 
any time without penalty. You have the right to refuse to participate in studies involving a particular 
method. You also have the right to refuse to answer particular questions. Your individual privacy will be 
maintained in all published and written information resulting from the study. 

If you have questions about your rights as a study participant, or are dissatisfied at any time with any 
aspect of this study, you may contact - anonymously, if you wish - the Institutional Review Board, 
Suburban Private University [contact information] . 

The study described above has been explained to me. My willingness to participate is indicated below. 
(Please check appropriate box.) 

I I I do not wish to participate . I do not wish to participate in this activity. In order for us to ensure 

that you will not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 

I I I wish to participate . I voluntarily consent to participate in this activity. I have had an opportunity 
to ask questions. I give permission for the study team to use my responses in research. (Please 
proceed to the following sections). 


I give consent to be interviewed and audiotaped during this study. (Please initial): Yes No 

I give consent for the responses I provide during the interviews and surveys, and the photographs taken 
during the observations to be used for research purposes in this study. (Please initial): Yes No 

I give consent to be observed and photographed during this study. (Please initial): Yes No 

I give consent for my academic records to be used for research purposes in this study. 

(Please initial): Yes No 

I have read the above and give my consent to participate in this study. 

NAME STUDENT ID# 

SIGNATURE DATE 

The extra copy of this consent form is for you to keep. 

Approval Date: 9/26/2003 

Expiration Date: 5/29/2004 
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NSF Center for the Advancement of Engineering Education 

CONSENT FORM: Longitudinal Study 

FOR QUESTIONS ABOUT THE STUDY, CONTACT: Project Researcher at Suburban Private 
University [contact information]. 

DESCRIPTION: 

You are invited to participate in a national research study funded the National Science Foundation (NSF), 
which aims to gain an in-depth understanding of how engineering students explore the educational 
opportunities available to them in college, plan and carry out their curriculums (successfully or 
unsuccessfully), and become engineers. The study will take place between Fall 2003 and Spring 2007. 
You will not need to do any more or less work in your classes because you have agreed to participate in 
this study, nor will you be asked to perform differently than you would if you did not participate. As 
discussed in the information meeting, a combination of the following methods will be used in the study to 
capture different aspects of your experience as a student who is planning to major in engineering: 

1) Interviews in which a trained observer will ask you standardized questions regarding your 
experiences and motivations as an undergraduate pre-engineering student. The interviews will be 
audiotaped. 

2) Internet-based questionnaires where you will be requested to provide some information about 
your experiences and motivations as an undergraduate pre-engineering student. 

3) A series of non-intrusive observations in which a trained researcher will accompany you for no 
more than 2 hours a week in various academic (e.g., lectures, study groups) and as well as nonacademic 
settings (e.g., dormitory, extracurricular activity settings) in order to obtain a general 

impression of undergraduate pre-engineering student life. The researcher will take notes, and if 
you permit, occasional photographs, in order to document his/her observations. The researcher 
might ask informal questions, which will be audiotaped if you agree. 

4) Access to your academic information maintained by the university registrar such as your SAT 
scores, high school GPA, college transcripts, gender, and ethnicity. This information will be kept 
in strictest confidence and will only be used for research purposes. 

All information collected will be kept confidential, and accessed only by authorized researchers who have 
signed a confidentiality agreement. To better preserve your privacy, a random identification code will be 
used in place of your name during the study. This is the same code that will be used in the interviews and 
questionnaires you have been asked to participate in. All documents that can be used to decrypt your 
identification code will be destroyed by 2012. The original data files (any audiotapes, survey responses, 
photographs) will be archived until the Faculty Investigator decides that they are no longer needed. 

RISKS : 

The potential risk is the inappropriate use of the information that is collected. To guard against this, the 
research personnel have signed a confidentiality agreement that limits access and use of data. (All 
encryption documents relating your randomly generated ID codes to your identity will be destroyed by 
December 2012. Data files will be archived until they are no longer needed.) Also, some people feel self- 
conscious when they are observed for the purposes of a study. 

BENEFITS: 

The study seeks to identify and characterize the pathways for making the choice of becoming an engineer. 
This understanding will enable the engineering community to pinpoint and clearly define areas in 
engineering education, where additional work is needed for improvement. 

PAYMENTS: 
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You will be compensated by check. Legally, you can be paid only if you are a US citizen, a legal resident 
alien (i.e., possess a "green" card), or have a work eligible visa sponsored by the paying institution. The 
amount of compensation will depend on your level of participation. Participants will be observed through 
a combination of any of the four methods described in the “description” section of this form, and 
compensated $175 per year. Annual payments will be made at the end of spring quarter upon continuous 
participation for that school year. There will be an equal number of participants in each group. However, 
if you chose to participate, it is not possible to know which group you will be in before the researchers 
obtain consent from all interested students. 

SUBJECT’S RIGHTS: 

If you have read this form and have decided to participate in this project, please understand that your 
participation is voluntary and you have the right to withdraw your consent or discontinue participation at 
any time without penalty. You have the right to refuse to participate in studies involving a particular 
method. You also have the right to refuse to answer particular questions. Your individual privacy will be 
maintained in all published and written information resulting from the study. 

If you have questions about your rights as a study participant, or are dissatisfied at any time with any 
aspect of this study, you may contact - anonymously, if you wish - the Institutional Review Board, 
Suburban Private University [contact information] . 

The study described above has been explained to me. My willingness to participate is indicated below. 
(Please check appropriate box.) 

I do not wish to participate. I do not wish to participate in this activity. In order for us to ensure 

that you will not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 

I wish to participate. I voluntarily consent to participate in this activity. I have had an opportunity 
to ask questions. I give permission for the study team to use my responses in research. (Please 
proceed to the following sections). 

I give consent to be interviewed during this study. (Please initial): Yes No 

I give consent to be audiotaped during this study. (Please initial): Yes No 

I give consent to be observed during this study. (Please initial): Yes No 

I give consent to be photographed during this study. (Please initial): Yes No 

I give consent for my academic records to be used for research purposes in this study. 

(Please initial): Yes No 

I give consent for the responses I provide during the interviews and surveys, and the photographs taken 

during the observations to be used for research purposes in this study. (Please initial): Yes No 

I have read the above and give my consent to participate in this study. 

NAME STUDENT ID# 

SIGNATURE DATE 

The extra copy of this consent form is for you to keep. 

Approval Date: 4/28/2006 

Expiration Date: 4/28/2007 
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LARGE PUBLIC UNIVERSITY 
GENERAL CONSENT EORM 
ACADEMIC PATHWAYS STUDY 

[list of investigators with contact information] 

INVESTIGATORS’ STATEMENT 
Purpose and Benefits 

We are engaged in an ongoing study to learn more about how engineering students explore the 
educational opportunities available to them in college, plan and carry out their curricula, and become 
engineers. We hope that the results of this study will help the engineering education community to 
improve the design of curriculum and teaching methods in order to help students to succeed in becoming 
engineers. You might not benefit directly from this study. 

Procedures 

This research is being conducted by researchers at Large Public University. Part of the data collection for 
the study involves observation and videotaping of the academic work of undergraduate engineering 
students. You will not need to do any more or less work in your classes because you have agreed to 
participate in this study, nor will you be asked to perform differently than you would if you did not 
participate. We are asking for your consent to collect the following forms of data: 

1. Videotape of your participation in today’s observational session. These videotapes will be kept by 
the researchers until December of 2012, to be used for research purposes only. 

2. Copies of written work, such as sketches and equations, that you produce or use during this 
observational session. 


All data collected will be kept confidential, and accessed only by authorized researchers who have signed 
a confidentiality agreement. Your name will not appear in transcriptions of the videotapes. If we publish 
the results of this study we will not use your name. 

You can request that we terminate the recording at any time and for any reason. You also have the option 
of reviewing the recording, and deleting all or part of it. 

We sometimes use recordings to make academic presentations. In the event that we plan to use a video 
recording of today’s session in such a presentation, we will mask your identity by electronic means. 

These electronic means will eliminate all instances in which non-subjects are directly identifiable. This is 
possible because we are using digital media to make these recordings, and these media make this masking 
process relatively easy. 


Risks, Stress, or Discomfort 

Some people might feel stress or discomfort as a result of the invasive nature of the research procedures. 

Some people might feel concerned that confidential information will be used inappropriately. To guard 
against this, all research personnel have signed a confidentiality agreement that limits access and use of 
data. 
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Some people might feel uncomfortable being observed and video recorded for the purposes of a study. 

We want to reiterate that you can request that we terminate the recording at any time and for any reason, 
and that you also have the option of reviewing the recording, and deleting all or part of it. 

Other information 

You must be at least 18 years of age to participate in this research. 

Participation in this study is voluntary. At any time during the study you may choose to stop participating. 
You can ask us to stop videotaping at any time, for any reason. You can ask us to terminate the 
observation at any time, for any reason. If you decide to stop participating, this will have no adverse 
effect on your course grades or your standing in school. 

If you have read this form and have decided to participate in this project, please understand that your 
participation is voluntary and you have the right to withdraw your consent or discontinue participation at 
any time without penalty. You have the right to refuse to participate in studies involving a particular 
method. Your individual privacy will be maintained in all published and written information resulting 
from the study. 
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SUBJECT'S STATEMENT 


The study described above has been explained to me. My willingness to participate is indicated below. 

(Please check appropriate box.) 

I I I do not wish to participate . I do not wish to participate in this study. In order for us to ensure that 
you will not be contacted again, please provide your name here: 

I I I wish to participate . I voluntarily consent to participate in this study. I have had an opportunity 
to ask questions. I give permission for the study team to use my responses in research. I 
understand that future questions I may have about the research or about my rights as a subject 
will be answered by one of the investigators listed above. (Please proceed to the following items 
1-3.) 

1 . I give consent to be observed during this study. I understand that some of my activity will be 
videotaped, and I consent to this. I understand that I will have the opportunity to review and edit 
the recording. I also understand that these tapes and photographs will be retained by the 
researchers until December, 2012, to be used for research purposes only. 

Please initial: Yes No 

2. I give consent for videotapes resulting from my participation in this study to be used in academic 
publications and presentations. 

Please initial: Yes No 

3. I give consent for samples of my written work to be used for research purposes in this study. 

Please initial: Yes No 


I have read the above and give my consent to participate in this study. 

NAME 

SIGNATURE 

DATE 

The extra copy of this consent form is for you to keep. 

cc: Participant 

Investigator's file 
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LARGE PUBLIC UNIVERSITY 
CONSENT FORM 

ACADEMIC PATHWAYS STUDY, ETHNOGRAPHIC GROUP 

[list of investigators and contact information] 

INVESTIGATORS’ STATEMENT 

We are asking you to take part in a research study. The purpose of this consent form is to give you the 
information you will need before you decide whether or not to take part in this study. Please read this 
form carefully. You may ask questions about what we will ask you to do, the risks, the benefits, your 
rights as a volunteer, or anything else about the research or this form that is not clear. When all your 
questions have been answered, you can decide if you want to take part or not. This process is called 
“informed consent.” We will give you a copy of this form for your records. 

Purpose of the Study 

We want to know more about how engineering students explore the educational opportunities available to 
them in college, plan and carry out their curricula, and become engineers. 

Procedures 

You have been selected as a participant in the Ethnographic group of the study. Data collection for the 
study will take place between Fall 2003 and Spring 2009, and will be conducted by researchers at Large 
Public University, Technical Public Institution, Urban Private University, and Suburban Private 
University. 

General Procedures: 


You will not need to do any more or less work in your classes because you have agreed to participate in 
this study, nor will you be asked to perform differently than you would if you did not participate. The 
following forms of data will be used to study different aspects of your experience as a student who is 
planning to major in engineering: 

1) Academic information maintained by the university registrar, including your college transcripts, 
SAT scores, high school GPA, gender, and ethnicity. This information will be kept in strictest 
confidence and will only be used for research purposes. 

2) Two 2-hour internet-based questionnaires, in which you will be asked to provide some 
information about your experiences and motivations as an undergraduate engineering student. 
Some questions on these questionnaires will be of a personal and potentially sensitive nature, 
including questions about your ethnicity, about your parents’ education and income levels, and 
about how much time you spend studying. You do not have to answer any questions you do not 
wish to answer. 

3) One formal audio taped interview per year for each of the three years of the study, each of which 
will take approximately 2 hours, in which a trained interviewer will ask a series of questions 
regarding your experiences and motivations as an undergraduate engineering student. Audio 
tapes of these interviews will be kept by the researchers for 10 years, until December of 2012, to 
be used for research purposes only. Some questions in these questionnaires will be of a personal 
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and potentially sensitive nature, including questions about your family background and its 
relationship to decisions you make about your education and career, about difficulties you might 
have in your coursework, and about how your relationships with friends, family, peers at school, 
and faculty relate to your performance in school and your career choices. You do not have to 
answer any questions you do not wish to answer. 

4) A series of observations in which a trained observer will accompany you for an average of 
approximately 2 hours per week in various academic settings (e.g., lectures, study groups) and 
academically-oriented extracurricular settings (e.g.. College of Engineering Sponsored 
workshops, engineering student society meetings) in order to obtain a general impression of 
undergraduate engineering student life. Some of these observations may be audio taped, video 
taped, and/or photographed. Audio tapes, video tapes, and photographs resulting from your 
participation will be kept by the researchers for 10 years, until December of 2012, to be used for 
research purposes only. 

5) Periodic brief interviews in which a trained observer, based on the observations made in 4) above, 
which will explore in more detail your experiences and motivations as an undergraduate 
engineering student. These interviews may be audio taped, video taped, and/or photographed. 
Audio tapes, video tapes, and photographs of these interviews will be kept by the researchers for 
10 years, until December of 2012, to be used for research purposes only. 

6) Samples of your written coursework, such as homework, project reports, and examinations. 

7) In some cases when research participants are no longer in the immediate area, or are unavailable 
to be present for a face-to-face interview, we will conduct interviews over the phone. These 
interviews will be recorded. 


Procedures for observing and recording activities: 

We will conduct observations only on campus and only of academically oriented activities. These can 
include both curricular (e.g., lectures, project meetings) and extracurricular (e.g., undergraduate 
engineering societies. College of Engineering sponsored workshops) activities. We anticipate video 
recording a small percentage of the activities that we observe - probably less than ten percent of the total 
observation time will be video recorded. 

You will have an important role in determining the time, place, and nature of the observation settings. 

We will consult with you each week on where and when that week’s observation will take place. We will 
solicit your opinion on what settings and what activities might be interesting and appropriate for us to 
observe. 

You will always know in advance that an observation will be taking place - we will not show up 
unannounced. A day before a scheduled observation, we will remind you via your preferred means of 
communication (e.g., phone or email) that the observation will take place. 

You will always know in advance whether or not we plan to tape during a given observation - we will not 
tape without having arranged this with you in advance. 

You will always have the option to ask us not to observe and/or to record a given activity. You may 
decline our suggestion to be observed and/or taped at a particular time or place, and you may cancel a 
scheduled observation for any reason and at any time, including after the observation has begun. In 
addition, you may ask us to discontinue taping for any reason and at any time. We will always comply 
with such requests immediately. 
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Procedures for storing and accessing data: 


All data collected, with the exception of video recordings, will he stored in a secure, password-protected 
database at Suhurhan Private University and will he shared with the researchers at the other 3 institutions. 

All data will he kept confidential, and accessed only hy authorized researchers at the four participating 
schools who have signed a confidentiality agreement. To better protect your privacy, we will generate a 
unique identification code that will be used in place of your name during the study. If we publish the 
results of this study we will not use your name. We will destroy the link between the data for this study 
and your name by December 2012. 

Whenever possible, we will remove identifying information from data collected from you. Your name 
and other identifying information will be removed from student records, surveys, transcripts of 
interviews, fieldnotes, and work samples. Removal of identifying information from these forms of data 
will take place as soon as possible after collection, and always before storage in the database. 

You will remain identifiable in some of the data that we will collect during this study. This includes 
audio and video recordings, and photographs. Information relevant to our procedures for identifiable data 
follows. 

Audio recordings and photographs will be stored in the database at SPri, and will be accessible to faculty 
and student researchers involved in the Academic Pathways Study. This includes researchers from Large 
Public University, Technical Public Institution, Urban Private University, and Suburban Private 
University. 

Video recordings will be stored in a locked cabinet in a private office in the LPub College of Education. 
Researchers at the other participating institutions will not have access to video recordings in which you 
are identifiable. 

You will have the option of reviewing all recordings in which you are identifiable (i.e., audio and video 
recordings, and photographs). You can request that we delete any such recording, or any part of it, and 
we will comply with this request. 

We sometimes use identifiable recordings in academic presentations. In such cases, we will offer you the 
option of reviewing recordings in which you are identifiable prior to making the presentation. This option 
will be in effect both during and after your participation in the study. 

Two faculty members in the LPub College of Engineering will have access to audio and video recordings 
and photographs in which you are identifiable. These researchers are subject to the same confidentiality 
agreement as are all other researchers associated with the Academic Pathways Study. 

Risks, Stress, or Discomfort 

Some people might feel discomfort or stress from the intensive nature of this research. Some people 
might feel that the research procedures are invasive. 

Some people might feel concerned that confidential information will be used inappropriately. To guard 
against this, the research personnel at each of the four participating institutions will sign a confidentiality 
agreement that limits access to and use of data. 
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Some people might feel concerned that their academic and professional careers might he negatively 
impacted as a result of the fact that some faculty in the College of Engineering have access to identifiable 
data. We want to stress that you have the option of reviewing all recordings in which you are identifiable, 
and that you have the option to ask that all or part of any recordings be deleted. We also want to stress 
that you have the option of terminating any observation and/or recording at any time and for any reason. 

Some people might feel uncomfortable when their responses to interviews are analyzed for the purposes 
of a study. 

Some people might feel uncomfortable being observed for the purposes of a study. 

Payments 

You will be compensated in cash. Legally, you can be paid only if you are a US citizen, a legal resident 
alien (i.e., possess a “green” card), or have a work eligible visa sponsored by the paying institution. You 
will be compensated at the rate of $175 per year, to be paid at the end of the Spring quarter. If you end 
your participation in the study early, you will be paid a pro-rated amount based on the duration of your 
participation. 

Benefits of the Study 

We hope that the results of this study will help the engineering education community to improve the 
design of curriculum and teaching methods in order to help students to succeed in becoming engineers. 
You might not benefit directly from this study. 

Other information 

Participation in this study is voluntary. At any time during the study you may choose to stop participating. 
A decision to stop participating will have no adverse effect on your course grades or your standing in 
school. 

This is a three-year study. We will contact you at the beginning of the Fall quarter each year to review 
with you the objectives and procedures of the study and to obtain signed consent for your continuing 
participation. 

If in the course of our observations we see evidence of suicidal tendencies or physical abuse, we are 
required to report this to appropriate authorities. All other identifiable information about you will be kept 
confidential. 

If in the course of our observations we recognize signs of unhealthy or dangerous behavior or 
psychological states, we will offer you information on where you can go to receive treatment. 

The following groups may need to review study records about you: Institutional oversight review offices 
at the research site, LPub, or state; and federal regulators. 


Termination of participation: 

As stated above, you are free to terminate your participation in this study for any reason and at any time. 
Under certain circumstances, it might be necessary for the research team to terminate your participation. 
Below are the conditions under and procedures by which this might occur: 
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If you leave Large Public University for any reason, including illness, transfer to another institution, etc., 
your participation in the study will be terminated. 

Our study is focused on engineering education, and we are accordingly recruiting subjects who display 
intent to major in engineering. We expect that some subjects might decide to major in a subject other 
than engineering. Such cases are quite relevant to the goals of our research, and we will as a result 
continue to include such subjects in our research for a period sufficient to capture the subject’s initial 
transition out of engineering. We expect this period to be at least one quarter and for up to a full 
academic year. We will terminate the subject’s participation in the study after this point. 

It is possible that you might, for a variety of reasons, become unwilling or unable to participate 
consistently in study activities, such as interviews and observations. In cases in which we have been 
unable to you consistently in study activities for a reasonable period of time, which we take to be 
approximately one month, we will approach you to inquire whether you wish to continue your 
participation. If you wish to continue, we will attempt to work out a satisfactory participation schedule. 
If after this point we continue to be unable to consistently engage you in study activities, we reserve the 
right to terminate your participation. Please note that termination of subjects for these reasons is an 
undesirable outcome for the study, and will be done only as a last resort. 


If you have read this form and have decided to participate in this project, please understand that your 
participation is voluntary and you have the right to withdraw your consent or discontinue participation at 
any time without penalty. You have the right to refuse to participate in studies involving a particular 
method. You also have the right to refuse to answer particular questions. Your individual privacy will be 
maintained in all published and written information resulting from the study. 

At the conclusion of your participation in the study, we will ask for permission to re-contact you should 
we wish to use identifiable recordings of you in public presentations. 
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SUBJECT'S STATEMENT 


The study described above has been explained to me. My willingness to participate is indicated below. 

(Please check appropriate box.) 

I I I do not wish to participate . I do not wish to participate in this study. In order for us to ensure that 

you will not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 

I I I wish to participate . I voluntarily consent to participate in this study. I have had an opportunity 
to ask questions. I give permission for the study team to use my responses in research. I 
understand that future questions I may have about the research or about my rights as a subject 
will be answered by one of the investigators listed above. (Please proceed to the following items 
1 - 6 .) 

1) I give consent for my academic records to be used for research purposes in this study. 

Please initial: Yes No 

2) I give consent to be interviewed during this study. I understand that these interviews will be 
audiotaped, and I consent to this. I understand that I will have the opportunity to review and edit 
these audiotapes. I also understand that these audiotapes will be retained by the researchers until 
December, 2012. 

Please initial: Yes No 

3) I give consent for the responses I provide during interviews and surveys to be used for research 
purposes in this study. 

Please initial: Yes No 

4) I give consent to be observed during this study. I understand that I can decline to be observed at 
any time and for any reason, including after an observation has begun. I understand that some of 
my activities will be audio taped, video taped, and/or photographed, and I consent to this. I 
understand that I can, at any time and for any reason, request that audio and video recordings be 
terminated. I understand that I will have the opportunity to review and edit these tapes and 
photographs. I also understand that recordings and photographs will be retained by the 
researchers until December, 2012. 

Please initial: Yes No 

5) I give consent for audiotapes, videotapes, and photographs resulting from my participation in this 
study to be used in academic publications and presentations. I understand that I will have the 
option of reviewing such materials before they are presented publicly. 

Please initial: Yes No 

6) I give consent for samples of my written coursework to be used for research purposes in this 
study. 

Please initial: Yes No 

7) I give consent for an interview to be conducted over the phone. I understand that this 
interview will be audio-recorded. 

Please initial: Yes No 
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I have read the above and give my consent to participate in this study. 

NAME SIGNATURE 

DATE 

The extra copy of this consent form is for you to keep. 

cc: Participant 

Investigator's file 

Ethnographic Consent Form (rev. 5/20/2008) 
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LARGE PUBLIC UNIVERSITY 
CONSENT FORM 

ACADEMIC PATHWAYS STUDY, OBSERVED GROUP 


[list of investigators and contact information] 

INVESTIGATORS’ STATEMENT 

We are asking you to take part in a research study. The purpose of this consent form is to give you the 
information you will need before you decide whether or not to take part in this study. Please read this 
form carefully. You may ask questions about what we will ask you to do, the risks, the benefits, your 
rights as a volunteer, or anything else about the research or this form that is not clear. When all your 
questions have been answered, you can decide if you want to take part or not. This process is called 
“informed consent.” We will give you a copy of this form for your records. 

Purpose of the Study 

We want to know more about how engineering students explore the educational opportunities available to 
them in college, plan and carry out their curricula, and become engineers. 

Procedures 

You have been selected as a participant in the Ethnographic group of the study. Data collection for the 
study will take place between Fall 2003 and Spring 2009, and will be conducted by researchers at Large 
Public University, Technical Public Institution, Urban Private University, and Suburban Private 
University. 

General Procedures: 


You will not need to do any more or less work in your classes because you have agreed to participate in 
this study, nor will you be asked to perform differently than you would if you did not participate. The 
following forms of data will be used to study different aspects of your experience as a student who is 
planning to major in engineering: 

1) Academic information maintained by the university registrar, including your college transcripts, 
SAT scores, high school GPA, gender, and ethnicity. This information will be kept in strictest 
confidence and will only be used for research purposes. 

2) Two 2-hour internet-based questionnaires, in which you will be asked to provide some 
information about your experiences and motivations as an undergraduate engineering student. 
Some questions on these questionnaires will be of a personal and potentially sensitive nature, 
including questions about your ethnicity, about your parents’ education and income levels, and 
about how much time you spend studying. You do not have to answer any questions you do not 
wish to answer. 

3) One formal audio taped interview per year for each of the three years of the study, each of which 
will take approximately 2 hours, in which a trained interviewer will ask a series of questions 
regarding your experiences and motivations as an undergraduate engineering student. Audio 
tapes of these interviews will be kept by the researchers for 10 years, until December of 2012, to 
be used for research purposes only. Some questions in these questionnaires will be of a personal 
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and potentially sensitive nature, including questions about your family background and its 
relationship to decisions you make about your education and career, about difficulties you might 
have in your coursework, and about how your relationships with friends, family, peers at school, 
and faculty relate to your performance in school and your career choices. You do not have to 
answer any questions you do not wish to answer. 

4) In some cases when research participants are no longer in the immediate area, or are unavailable 
to be present for a face-to-face interview, we will conduct interviews over the phone. These 
interviews will be audio-recorded. 


Procedures for storing and accessing data: 

All data collected, with the exception of video recordings, will be stored in a secure, password-protected 
database at Suburban Private University and will be shared with the researchers at the other 3 institutions. 

All data will be kept confidential, and accessed only by authorized researchers at the four participating 
schools who have signed a confidentiality agreement. To better protect your privacy, we will generate a 
unique identification code that will be used in place of your name during the study. If we publish the 
results of this study we will not use your name. We will destroy the link between the data for this study 
and your name by December 2012. 

Whenever possible, we will remove identifying information from data collected from you. Your name 
and other identifying information will be removed from student records, surveys, and transcripts of 
interviews. Removal of identifying information from these forms of data will take place as soon as 
possible after collection, and always before storage in the database. 

You will remain identifiable in some of the data that we will collect during this study. Information 
relevant to our procedures for identifiable data follows. 

Audio recordings and photographs will be stored in the database at SPri, and will be accessible to faculty 
and student researchers involved in the Academic Pathways Study. This includes researchers from Large 
Public University, Technical Public Institution, Urban Private University, and Suburban Private 
University. 

You will have the option of reviewing all audio recordings in which you are identifiable. You can request 
that we delete any such recording, or any part of it, and we will comply with this request. 

Two faculty members in the LPub College of Engineering will have access to audio recordings in which 
you are identifiable. These researchers are subject to the same confidentiality agreement as are all other 
researchers associated with the Academic Pathways Study. 


Risks, Stress, or Discomfort 

Some people might feel discomfort or stress from the intensive nature of this research. Some people 
might feel that the research procedures are invasive. 

Some people might feel concerned that confidential information will be used inappropriately. To guard 
against this, the research personnel at each of the four participating institutions will sign a confidentiality 
agreement that limits access to and use of data. 
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Some people might feel concerned that their academic and professional careers might he negatively 
impacted as a result of the fact that some faculty in the College of Engineering have access to identifiable 
data. We want to stress that you have the option of reviewing all recordings in which you are identifiable, 
and that you have the option to ask that all or part of any recordings be deleted. We also want to stress 
that you have the option of terminating any observation and/or recording at any time and for any reason. 

Some people might feel uncomfortable when their responses to interviews are analyzed for the purposes 
of a study. 

Payments 

You will be compensated in cash. Legally, you can be paid only if you are a US citizen, a legal resident 
alien (i.e., possess a “green” card), or have a work eligible visa sponsored by the paying institution. You 
will be compensated at the rate of $175 per year, to be paid at the end of the Spring quarter. If you end 
your participation in the study early, you will be paid a pro-rated amount based on the duration of your 
participation. 

Benefits of the Study 

We hope that the results of this study will help the engineering education community to improve the 
design of curriculum and teaching methods in order to help students to succeed in becoming engineers. 
You might not benefit directly from this study. 

Other information 

Participation in this study is voluntary. At any time during the study you may choose to stop participating. 
A decision to stop participating will have no adverse effect on your course grades or your standing in 
school. 

This is a three-year study. We will contact you at the beginning of the Fall quarter each year to review 
with you the objectives and procedures of the study and to obtain signed consent for your continuing 
participation. 

If in the course of our interviews we see evidence of suicidal tendencies or physical abuse, we are 
required to report this to appropriate authorities. All other identifiable information about you will be kept 
confidential. 

If in the course of our interviews we recognize signs of unhealthy or dangerous behavior or psychological 
states, we will offer you information on where you can go to receive treatment. 

The following groups may need to review study records about you: Institutional oversight review offices 
at the research site, LPub, or state; and federal regulators. 


Termination of participation: 

As stated above, you are free to terminate your participation in this study for any reason and at any time. 
Under certain circumstances, it might be necessary for the research team to terminate your participation. 
Below are the conditions under and procedures by which this might occur: 
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If you leave Large Public University for any reason, including illness, transfer to another institution, etc., 
your participation in the study will be terminated. 

Our study is focused on engineering education, and we are accordingly recruiting subjects who display 
intent to major in engineering. We expect that some subjects might decide to major in a subject other 
than engineering. Such cases are quite relevant to the goals of our research, and we will as a result 
continue to include such subjects in our research for a period sufficient to capture the subject’s initial 
transition out of engineering. We expect this period to be at least one quarter and for up to a full 
academic year. We will terminate the subject’s participation in the study after this point. 

It is possible that you might, for a variety of reasons, become unwilling or unable to participate 
consistently in study activities, such as interviews and observations. In cases in which we have been 
unable to you consistently in study activities for a reasonable period of time, which we take to be 
approximately one month, we will approach you to inquire whether you wish to continue your 
participation. If you wish to continue, we will attempt to work out a satisfactory participation schedule. 
If after this point we continue to be unable to consistently engage you in study activities, we reserve the 
right to terminate your participation. Please note that termination of subjects for these reasons is an 
undesirable outcome for the study, and will be done only as a last resort. 


If you have read this form and have decided to participate in this project, please understand that your 
participation is voluntary and you have the right to withdraw your consent or discontinue participation at 
any time without penalty. You have the right to refuse to participate in studies involving a particular 
method. You also have the right to refuse to answer particular questions. Your individual privacy will be 
maintained in all published and written information resulting from the study. 

At the conclusion of your participation in the study, we will ask for permission to re-contact you should 
we wish to use identifiable recordings of you in public presentations. 
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SUBJECT'S STATEMENT 


The study described above has been explained to me. My willingness to participate is indicated below. 

(Please check appropriate box.) 

I I I do not wish to participate . I do not wish to participate in this study. In order for us to ensure that 

you will not be contacted again, please provide your name here: 

(Please do not read any further. Thank you for your time and consideration.) 

I I I wish to participate . I voluntarily consent to participate in this study. I have had an opportunity 
to ask questions. I give permission for the study team to use my responses in research. I 
understand that future questions I may have about the research or about my rights as a subject 
will be answered by one of the investigators listed above. (Please proceed to the following items 
1 - 6 .) 

1) I give consent for my academic records to be used for research purposes in this study. 

Please initial: Yes No 

2) I give consent to be interviewed during this study. I understand that these interviews will be 
audio-recorded, and I consent to this. I understand that I will have the opportunity to review and 
edit these audio files. I also understand that these audiotapes will be retained by the researchers 
until December, 2012. 

Please initial: Yes No 

3) I give consent for the responses I provide during interviews and surveys to be used for research 
purposes in this study. 

Please initial: Yes No 

4) I give consent for audiotapes resulting from my participation in this study to be used in academic 
publications and presentations. 

Please initial: Yes No 

5. I give consent for an interview to be conducted over the phone. I understand that this 
interview will be audio-recorded. 

Please initial: Yes No 


I have read the above and give my consent to participate in this study. 

NAME SIGNATURE 

DATE 

The extra copy of this consent form is for you to keep. 

cc: Participant 

Investigator's file 

Observed Group Consent Form (rev. 5/20/2008) 
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Appendix 1-E 

APS Longitudinal Cohort Demographics Table 


This table presents status and demographic details for each ID number issued in the APS Longitudinal Cohort. Data in the table was 
obtained from the following sources: 

• Citizenship status was determined from PIE survey responses. In two cases, different responses were recorded at different time 
points, and these are explained in the far right column. Citizenship was asked on surveys #1, #4 and #7. 

• Ethnicity was determined from PIE responses. In cases where different responses were recorded at different time points, the 
earliest response was used. Ethnicity was asked on surveys #1, #4 and #7. 

Survey 1 response was used if subject started the study in Year 1. 

Survey 4 response was used if subject started the study in Year 2, or did not respond to the ethnicity question on survey 1 . 

• The year in which students left the study was determined from the date of the last piece of data we collected for that student. 

• Engineering Persistence was determined from the major listed on the last academic transcript supplied to APS, or from records 
supplied by the department or institution if transcripts were not available or complete. 

Persister = documented engineering major 

Non-persister = documented non-engineering major OR interview data (exit interview or regular APS interview) where 
subject states he/she will not pursue engineering. Non-persisters had an interest in an engineering major upon enrolling in 
University. 

Eost to follow-up = did not complete APS and major status not known 

• Year 4 Transcripts were supplied by the institutions. 

• East Recorded Major was determined from transcripts, if available, or from official department or school records 
A key to notations in the table is included on the last page. 
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Puerto Rican 
Mex = Mexican 
Lat = Other Latino 
W = White 

Other = other race/ethnicity 
Muiti = muitipie races/ethnicities 


SCH4STU35 

L 

1 

Asian 

Y3 

1 

NP 

SCH4STU36 

L 

1 

Asian 



P 

SCH4STU37 

L 

1 

W 

Y3 

1 

NP 

SCH4STU38 

L 

1 

Asian 

Y3 

1 

NP 

SCH4STU39 

L 

1 

W 



P 

SCH4STU40 

L 

1 

Asian 



P 


Initiai ethnicity response is used when different response was given iater. 


P (Persister) = subjects with documented engineering major at end of Y4 (from academic transcript or from department) regardiess of whether or 
not they suppiied Y4 data 

N (Non-persister) = subjects with documented non-engineering major OR who compieted an exit interview or reported during a reguiar interview 
that they wouid not pursue an engineering major 
O (other) = subjects meeting neither persister nor non-persister criteria 
L (Lost to foiiow-up) = did not compiete APS; reason unknown; may have ieft the insititution 
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Appendix 2-A 

Original APS Longitudinal Cohort (Cohort 1) Sampling Plan 

Introduction 

The goal of the Longitudinal Cohort is to identify and characterize the pathways for making the choice of 
hecoming an engineer. Incoming freshman will he studied through ethnographic, interview and survey 
techniques up until the end of their junior year. There will he 160 participants total, 40 participants each 
from TPuh, UPri, SPri, and LPuh. In each institution, 8 of the 40 will he studied in depth though 
ethnographies as well interviews and surveys. Since ethnographic studies are much more resource 
intensive, the other 32 will he studied though interviews and surveys only. In addition, there will he a 
control group at each institution consisting of 40 participants, who will not he subjected to any form of 
direct observation. Key statistics such as SAT scores, transcripts, major status, and other material 
institutional statistics their institutions tracks and keeps on file in the registrar’s office will be monitored 
for both groups. 

Institutional Background Information 


Technical Public Institution (TPub) 

Approximately 600 - 650 engineering freshmen enroll at TPub each year. The students are essentially 
identified as engineering students since TPub is an engineering institution, however students do not 
necessarily have to declare their major during their freshman year. Some students declare when they 
arrive, others declare later during their matriculation. All freshmen must enroll in “freshmen success” and 
their instructor for the course is also their advisor. Each instructor/advisor has 10-15 advisees. 

Engineering majors offered at TPub included in this study are Chemical Engineering, (General) 
Engineering, Geology and Geological Engineering, Geophysics, Metallurgical and Materials Engineering, 
Mining Engineering, Petroleum Engineering, Engineering Physics. Eor the purposes of this study the 
following majors will be excluded from the study: Economics, Mathematical and Computer Science, and 
Chemistry. In 2002, over 500 students graduated from the undergraduate engineering programs at TPub 
with over 400 students receiving degrees in the majors not excluded from this study. 

Urban Private University (UPri) 

Approximately 1 80 freshmen enter the UPri engineering program each year. Ereshmen are accepted into 
the engineering program upon enrolling at UPri, and all freshmen across the University are required to 
stay on campus. Engineering majors offered at UPri include Chemical, Civil, Systems and Computer 
Science, Mechanical, and Electrical Engineering. In 2002, over 100 students graduated from 
undergraduate engineering programs at UPri. 

Suburban Private University (SPri) 

Approximately 320-350 freshmen self-identify as interested in engineering at SPri each year. Ereshmen 
designate a preliminary academic interest (PAI) in Engineering, including Computer Science, during the 
summer and are assigned Academic Advisors that will advise them until they declare a major. Their 
selection of PAI does not guarantee or bind students to declare engineering. Students at SPri do not 
usually declare their major until the end of their sophomore year or during their junior year. The 
University does not offer a traditional introduction to engineering course, but does offer a few engineering 


APS Research Processes and Procedures 
January 2009 


2A-1 



related seminars for freshman with limited enrollment, approximately 15 students. Freshmen have to take 
a yearlong sequence in the humanities and a two-quarter course in writing and rhetoric. During the 
remainder of the freshmen year, students interested in engineering usually opt to enroll in math courses 
(either single or multi-variable calculus), start the physics sequence, and take a freshman seminar. The 
School of Engineering offers the following undergraduate engineering majors: Chemical Engineering, 
Civil and Environmental Engineering, Electrical Engineering, Material Science & Engineering, 
Management Science and Engineering (Industrial Engineering), Mechanical Engineering, Computer 
Science, Engineering/General Engineering- Aeronautics/ Astronautics, Computer Systems Engineering, 
Individually Designed Majors, Product Design, BioMedical Computation, and Bio Mechanical. Eor the 
purposes of this study. Computer Science will be excluded. In 2002, over 300 students graduated from 
undergraduate engineering programs at SPri, including 154 students in Computer Science. 

Large Public University (LPub) 

Approximately 650 freshmen are coded as pre-engineering prior to their arrival at LPub. Students can 
change this coding at any time and it does not bind the student to apply to the engineering program. Either 
the student or the University may assign students with the “pre-engineering” code. An additional small 
group of students, approximately 20, are admitted into an engineering major as incoming freshmen. Pre- 
engineering students may apply to multiple engineering majors granted they have completed the requisite 
pre-engineering coursework. Students usually begin applying at the end of their sophomore year and are 
admitted into the engineering major beginning with their junior year. Admission into the undergraduate 
engineering program is extremely competitive and not all applicants are admitted into the engineering 
program. LPub offers an Introduction to Engineering Design course (Engr 100) as a recommended course 
for pre-engineering students. The typical first quarter course load for a pre-engineering student is 
calculus, inorganic chemistry, and either English Composition or Engr 100. The College of Engineering 
offers the following undergraduate engineering majors: Aeronautics and Astronautics Engineering, 
Bioengineering, Chemical Engineering, Civil and Environmental Engineering, Computer Engineering, 
Electrical Engineering, Materials Science and Engineering, Mechanical Engineering, Technical 
Communication and Interdisciplinary Engineering Studies. In 2002, over 600 students graduated from 
undergraduate engineering programs at LPub. 

Identifying Students for the Study 

There will be 160 observed participants total in the Longitudinal Cohort: 40 participants each from TPub, 
UPri, SPri, and LPub. An additional 160 participants, 40 at each institution, will be identified to comprise 
a control group at each school. The observed and control students will simultaneously be identified based 
on the criterion at each school and will randomly be assigned to either the observed or control group. 

Each institution will also oversample from its respective underrepresented populations of engineering 
students. The percent ratio of participants identified below will be representative of both the observed and 
control group. 


Technical Public Institution 

Students will be selected to participate in the study based on the following criteria: 

• Gender: only 23% of TPub undergraduates are women so this population will be oversampled to 
obtain a 50/50 percent ratio of women to men in the study. 
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• Underrepresented Ethnic Populations: only 1 1% of TPub undergraduates are minority students, 
including African-American, Asian American, Hispanic American, and Native- American 
populations. For the purposes of this study, African-American, Hispanic American, and Native 
American populations will he oversampled. Another suh-group of underrepresented populations 
will include participants in the TPub MEP summer program. The TPub MEP summer program is 
a remedial program some students must attend and pass in order to gain admission into TPuh. 
(Targeted percent ratio to he around 25/75 underrepresented populations to majority populations.) 

• Keen Interest in Engineering - interviews will he conducted with students to assess their interest 
in engineering (targeted percent ratio to he determined). 

• Social Integration with Campus - based on focus groups of upper-class students, social support 
groups are important to a successful matriculation through TPub. The study will include students 
from a range of residences (on and off-campus), varsity athletes, and students involved in other 
TPub activities (targeted percent ratio to be determined). 

Urban Private University 

Students will be identified to participate in the study during the summer as soon as their acceptance into 
the engineering program has been confirmed by the payment of their enrollment fee. The freshman 
engineering class is relatively small, averaging 149 US and 35 Non-US students. Thus, the main criteria 
for participating in the study are: 

• Gender - the School of Engineering has a small representation of men, so this population will be 
oversampled to obtain a 50/50 percent ratio of men to women in the study. 

• Origin - the school of engineering has a large representation of international students and this 
population will be oversampled to obtain a 50/50 percent ratio of US to Non-US students. 

• Participation m freshman summer bridge program - a select group of students is invited to 
participate in the Summer Bridge program based on their SAT scores. At the end of the program, 
it is expected that students will increase awareness of the requirements and rewards of the 
engineering professions; proficiency to earn above average grades during the freshmen year; and 
knowledge of campus resources. This population will be sampled to obtain a 50/50 percent ratio 
of participants in the Summer Bridge program to non-participants. 

Suburban Private University 

Students will be identified to participate in the study during mid-June - July. Freshmen are required to 
submit several information forms and questionnaires by mid-June. These forms include personal 
information, information about their academic interests, as well as housing preferences. Students will be 
selected to participate in the study based on the following criteria: 

• Preliminary Academic Interest (PAI) in Engineering (target 100 percent of participants) 

• Gender - the freshmen engineering class has a small representation of women (approximately 
25%), so this population will be oversampled to obtain a 50/50 percent ratio of women to men in 
the study 

• Underrepresented Ethnic Populations: African Americans, Native Americans, Mexican 
Americans, Puerto Ricans, and other Latino groups comprise approximately 25% of the freshmen 
engineering class. We will oversample from these populations to ensure at least a 25/75 percent 
ratio of underrepresented ethnic populations to majority populations. 

• Participation in SPri summer bridge program - a selected group of around 40 students from 
underrepresented ethnic populations as well as women. SSEA enables students to explore various 
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engineering disciplines and science and engage with engineering faculty early in their academic 
career. We would like to have approximately 25/75 percent ratio of SSEA participants to non- 
participants expressed as an equal 25/75 percent ratio of women and men SSEA participants to 
women and men non-participants. 

• Housing assignments (4 class dorms, all freshman dorms, Ereshman Sophomore College, ethnic 
theme houses) and participation in varsity athletics are two additional factors we would like 
account for in both the observed and control group (targeted percent ratios to be determined). 

Large Public University 

LPub will seek to identify students who during their freshman year are participating in activities that (in 
retrospective studies) indicate an interest in being admitted to the engineering majors. Since our goal is to 
understand both the pathways into engineering and those that fail to flow into the field, our primary goal 
is to identify students whose activities display organized intent. In other words, we will not be selecting 
our sample from only those students that indicators suggest are very likely to succeed and be retained in 
engineering, on the basis of either academic or cultural factors. 

The criteria therefore for identifying between 75%-100% of the Longitudinal Cohort will be the 
following: 

• Students Coded as “Pre-Engineering” 

• Erom those coded as pre-engineering, we will then select from among students enrolled during 
the first quarter in one or more prerequisite courses required for all engineering majors. Students 
enrolled in one of the prerequisite math courses (1 in a sequence of 3 calculus courses) and the 
chemistry course (inorganic chemistry) will be included in the possible sample. Students also 
enrolled in Engineering 100 will be preferred in the sample. 

• Upon invitation to the study, students express an evident willingness to participate in the study for 
three years. 

The criterion for identifying up to 25% of the Longitudinal Cohort will be students who are among a 
small number admitted to an engineering major at LPub as incoming freshman. Approximately 2% of the 
overall engineering undergraduate population (20 students) enters the engineering majors through this 
route. Demographically, most of these students are male and are either of Caucasian or Asian- American 
descent. We will select up to 25% of the cohort (10 students) from this group, with the determining factor 
being the number of female students from this group that we can enlist into the study. Eor example, if 
there are three women who enter the major this way and are willing to participate, we will recruit three 
men from this group. The purposes of studying this more “elite” sub-population are two-fold. Eirst, we 
want to see if their pathways differ significantly from those in the more typical track. Second, we want to 
be able to understand how women and men in this more elite track navigate their education in comparison 
to those in the typical track. 

Since only about 20% of the freshmen meet the above criteria 1 and 2, we will be oversampling from this 
population. 

Additional factors considered in the design of the observed and sample groups include: 

• Gender - in 2002, around 26% of graduates of the engineering program were women so this 
population will be oversampled to obtain a 50/50 percent ratio of women to men in the study. 

• Underrepresented Ethnic Populations - in 2002, around 6% of graduates of the engineering program 
were minority students, including African-American, Hispanic American, and Native- American 
populations. These populations will be oversampled to obtain a 25/75 percent ratio of 


2A-4 


APS Research Processes and Procedures 

January 2009 



underrepresented to majority populations. Some of these students may be identified from participation 
in MESA activities during their high school years and who meet the above described criteria (MESA 
stands for Math, Engineering, and Science Academy, a program designed to recruit underrepresented 
populations). 

Inviting Students to Participate in the Study 


Technical Public Institution 

The details for inviting students to participate in the study at TPub are being finalized. Some strategies for 
recruiting students to participate in the study include reviewing information garnered from a survey or 
questionnaire, conducting interviews with students, and contacts in their Ereshmen Success course. 


Urban Private University 

A personalized letter will be sent to identified students in July to their permanent address. The letter will 
attempt to persuade students to participate in the study, and give descriptive information about the nature 
of the study. The letter will also include a consent form for the students to complete with a stamped- 
returned envelope. A follow-up phone call will be placed to further explain the study and answer any 
questions the students might have about the study. 

Suburban Private University 

A personalized letter sent to identified students in July - August to their permanent address. The letter will 
include enough information to garner interest in the study but not persuade students to study engineering, 
and a consent agreement with a stamped-returned envelope. Subsequent communications before 
interested students arrive on campus will establish a short meeting with a member of the longitudinal 
team once on campus to secure their participation in the study. Other venues for advertising the study 
include introductory mathematics courses, engineering programming during the New Student Orientation, 
their academic advisors, the Engineering Diversity Programs office, residential staff, and engineering 
societies. 

Large Public University 

A personalized letter sent in October to their university address inviting them to participate in the study. 
Members of the longitudinal research team will also make short presentations in the calculus sequence 
courses. The in-class solicitation and letter will include enough information to garner interest in the study 
but not to persuade students to study engineering. Subsequent communications with students will include 
a short individual meeting with a member of the research team to secure their participation and explain 
the commitment. Other venues for advertising the study include introductory mathematics courses, the 
chemistry course, the engineering presentations during the New Student Orientation, their academic 
advisors. Engineering 100 course, and a list-serv accessible to students coded as pre-engineering majors. 

Replacement Strategy 
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We will choose a cohort of 40 freshmen to follow through their first three years at their respective 
institutions. At TPuh and UPri, this represents following students through 2-3 years in their engineering 
major, while at SPri and LPuh, students will he followed into the end of their first year in an engineering 
major. There are multiple pathways into engineering other than clear and linear progress from first quarter 
freshman year. If however all 160 observed students remain in engineering through the junior year, we 
will likely have chosen our initial sample in a way that is somewhat unrepresentative of the overall 
populations of engineering students. Also there are attrition rates at each school, for instance at LPuh, on 
average only about 75% of applicants are admitted into the engineering program each year. Because we 
are seeking relative representation of pathways, we will therefore employ an ethnographically informed 
strategy for bringing students into the sample as others drop out. As students drop out and as we learn 
more about the other pathways into the disciplines (e.g., community college transfers, switching from a 
more basic science major or mathematics), we will solicit participation from students who have these 
basic profiles. At the end of the 3 years, we should therefore have a solid and closely observed image of 
what the first three years are like for a large number of students, but we will also have some solid 
understandings of other pathways based on replacement participants. 
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Appendix 2-B 

Sample APPLES Institution Overview and Recruitment Plan 


APPLES Institution Overview Checklist 


The APPLES Institution Overview Checklist serves three central purposes, it assists us 
in: (1) ensuring that the relevant populations of students will be sampled at your 
institution, (2) establishing the target sample size for your institution - including the best 
ways to meet the strata (or sub-group) targets and finally (3) accurately interpreting data 
from your institution. 

Date due: 15 September 2007 - Please return this form to us by then! 

If you have any questions, please email your APPLES liaison or info@applesurvev.org 




Coordinator information: Please provide your institution’s APPLES 
coordinator’s name, contact information and best means to contact him/her. 


Example: 


Coordinator.- Dr. Alexandria Smith, Director of Institutional Research 

Em ai I : asmith@orchard. edu 

Phone: 333-222-5555 

Fax: 333-222-5554 

Best way to contact: Email 


I I 2. Overview description: Please provide a short description (1 brief paragraph) of 

your university. 

Orchard is a private research university in a suburban setting located in the eastern 
United States, with an enrollment of about 14,000 students, divided equally between 
graduate and undergraduate students. Orchard’s School of Engineering is one of three 
schools at Orchard that offers an undergraduate degree. The University attracts students 
from around the nation and the world, with fifty percent of students classified as non- 
Caucasian. Of the approximately 1600 freshmen entering each year, 550 to 620 tend to 
self -identify as being interested in engineering. (Entering freshmen do not formally 
declare majors.) 

□ 3. List of engineering majors: Please provide a list of your institution’s 
engineering majors (as defined by your institution). 

Orchard’s: chemical engineering, civil engineering, computer science, electrical 
engineering, mechanical engineering, management engineering, materials science, 
industrial design, and environmental engineering. (In the enrollment chart, industrial 
design and environmental engineering are counted as “Engineering”) 


' ■ This material is based upon work supported by the United States Nationai Science Foundation under Grant No. ESI-0227558. 
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4. List of technical non-engineering majors: Please provide a list of technical 
non-engineering majors at your institution. We define technical non-engineering 
majors as non-engineering majors that share some of the mathematical, scientific 
and/or technological aspects of an engineering major, e.g. majors that: 

• share engineering prerequisites 

• many students who started off in engineering later migrated to 

• produce graduates who may take jobs similar to engineering graduates. 

At Orchard: Applied Mathematics, Economics, Physics, Management Sciences. 

(Economics at Orchard is highly analytical) 


5. Overview of major declaration process: Please provide a brief description of 
how and when students declare an engineering major at your institution. 

Students apply to Orchard without specifying an intended major. They must declare their 
major by the first quarter of their junior year or by the time they have completed 85 units. 
At Orchard we are able to get an estimate of the number of incoming potential Freshmen 
engineering students from a form they fill out (after being admitted) where we ask their 
preliminary area of academic interests for the purpose of assigning them an academic 
advisor. 


6. Engineering enroWment and graduation figures: Please provide the most 
recent enrollment and graduation numbers for your School of Engineering. (Your 
institution’s current figures are preferable to existing ASEE figures). 


ORCHARD’S ENGINEERING 
ENROLLMENT - Fall 2007 

Frosh 

1st 

Year 

Soph 

2nd 

Year 

Junior 
3rd Year 

Senior 
4th Year 

Part 

Time 

Total 

Full 

Time 

Total 

Chemical Engineering (B.S.) 

0 

5 

16 

24 

0 

45 

Civil Engineering (B.S.) 

0 

3 

17 

29 

0 

49 

Undeclared 

605 

547 

100 

10 

0 

1262 

Computer Science (B.S.) 

0 

17 

44 

84 

0 

145 

Electrical Engineering (B.S.) 

0 

9 

38 

71 

0 

118 

Engineering (B.S.) 

0 

2 

46 

58 

0 

106 

Management Engineering (B.S.) 

0 

6 

31 

74 

0 

111 

Materials Science (B.S.) 

0 

0 

1 

2 

0 

3 

Mechanical Engineering (B.S.) 

0 

4 

37 

63 

0 

104 

TOTAL 

605 

593 

330 

415 

0 

1943 

Frosh and Soph engineering majors largely estimated from students' preliminary areas of 
academic interest at acceptance to Orchard. 


GRADUATION FIGURES: In 2007, 89% of Orchard’s declared engineering students 
graduate within four years of enrolling at Orchard. 


Source: Orchard’s Registrar’s Office 


* *' This material is based upon work supported by the United States Nationai Science Foundation under Grant No. ESI-0227558. 
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7. University enroWment and graduation figures: Please provide the most recent 
undergraduate enrollment and graduation numbers for your institution. 


ORCHARD UNDERGRADUATE 
ENROLLMENT - Fall 2007 

Frosh 

Soph 

Junior 

Senior 

Total 

All Undergraduates 

1623 

1689 

1598 

1678 

6588 


Source: Orchard’s Registrar’s Office 


8. Unique sub-popuiations: Does your institution have a (or several) unique sub- 
population of students? APPLES tracks gender, ethnicity, enrollment status 
(part-time versus full-time), and international students. Examples of other unique 
sub-populations might be a high commuter population or a large number of “non- 
traditional” students. 

At Orchard, we have a targe number of internationat students. Atmost ait of our 
undergraduates are 18-22 years otd and five on campus. 


9. Choose a deployment week: From the list below please rank your preferences 
for the week APPLES will be deployed at your institution (and note if any of the 
dates would not work at all.) 

• 28 January-1 February 

• 11-15 February 

• 25-29 February 

Things to consider: (1) Your schedule as additional recruitment during the week 
of deployment may require extra time from you. (2) Your institution’s schedule, 
e.g. other surveys that your institution may be taking part in, holidays or special 
events. In our experience, the end of the term is not a good time to deploy a 

surve y - the beginning seems to be fine. 

(1) 28 January-1 February, (2) 25-29 February, (3) 1 1-15 February. Option 3 is highty 
undesirabie for us because we have “Engineering Week’’ that week - most of our 
engineering students wiii be off campus doing different projects and may not be as iikeiy 
to participate in an oniine survey. 


10. Significant structure changes? In the last year, have there been any 
significant structural, curricular, or institutional changes at your university that 
would impact the profile of your student body? 


We are in the process of reorganizing our engineering curricuium to better meet specific 
ABET criteria - we do not however, expect this to noticeabiy impact our students. 



This material is based upon work supported by the United States Nationai Science Foundation under Grant No. ESI-0227558. 
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Orchard - APPLES Recruitment Plan 


This is the iast assignment we wiii ask you to do for us before the APPLES depioyment on your 
campus in eariy 2008. We estimate it wiii take no more than one hour to compiete. The information 
beiow wiii assist us and you in recruiting the needed subject participation in APPLES to achieve 
meaningfui resuits. If you have any questions, piease emaii your APPLES iiaison. This is due: 1 5 
November 2007. Thanks! 

Coordinator: Dr. Robert Jones, Associate Dean, School of Engineering 
Contact; robert.jones@orchard.edu 

Recruitment classification: SMALL (> 500 undergraduate engineering students) 
Deployment week: 28 January - 1 February (Your 1 choice) 

Your campus survey URL: http://orchard.applesurvey.org 

(1) How many APPLES posters would you like us to send you? 

See attached - they are 11 ” x 1 7” and will be tailored to your institution. We are 
able to send you up to 20 posters. 

(2) Choose one of the suggested pseudonyms below for your institution. 

In all external publications discussing APPLES data or findings (except your 
institution’s final APPLES report), pseudonyms will be used to refer to 
participating institutions. 

□ Connerman University 

□ State Tech University 

□ Midland University 

(3) Complete the shaded Orchard columns for Recruitment Plans A and B. 

Please see “Notes on Recruitment Plans and APPLES Deployment” (page 3) for 
instructions and additional information. The “Orchard” column of the Recruitment 
Plan provides sample recruitment methods for each of the stratum. 
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Plan A - Planned Recruitment for APPLES Subjects 


Strata (min target) 

Response rate 
needed at 
Orch 

Orchard 

Grove {sample) 

All (50) 

16-"% 


Email to entire 
undergraduate 
engineering list 
Posters - around 
campus 

Engineering students 
(40) 

16% 


Same as above. 

Non-persisters (10) 

[students who initially 
intended to study 
engineering but opted 
for a non-engineering 
major] 



Targeted email to 
students majoring in 
non-engineering fields 
(i.e., chemistry, math, 
economics departmental 
email lists) 


Plan B (Targeted recruitment if minimum responses are not achieved) 


Strata (min target) 

Response 
rate needed 
at Orch 

Orchard 

Grove {sample) 

Female students (10) 

? 


Email to SWE members 

Male students (10) 

4% 


(won’t be a problem) 

Minority students 

(0) 

n/a 


Email to Minority 
Engineering program 

International students* 
(10) 


Let's discuss the International 
students 

Email to International 
Student Center 
distribution list 

Transfer students (?) 


Let's discuss 

Email to transfer 
students 

Freshmen (10) 

9% 


Email to freshmen 

Sophomore (10) 

16% 


Email to sophomores 

Juniors (10) 

29% 


Email to juniors 

Seniors (10) 

22% 


Email to seniors 

Non-persisters (still 
can’t get enough...) 



Eollow-up email to 
students majoring in 
non-engineering fields 
(i.e., chemistry, math, 
economics) 

Specific engineering 
major 



Targeted email to 
students in 
underrepresented 
major(s) 

Part-time students 

n/a 


NA 

* Denotes special sub-populations at ORCH: International exchange stuc 

ents. 
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Notes on the Recruitment Plans and APPLES 
Deployment at Orchard 

Orchard's strata targets for APPLES are listed in the recruitment plans. These numbers 
are based on your institution’s overall and engineering undergraduate enrollments, and the 
requirements for statistical analysis given the other institutions taking part in APPLES. 

Based on the required response rates, we may need to put extra effort into recruiting juniors, 
seniors and women at Orchard. We need a minimum of ten International respondents in 
order to do statistical analysis on that stratum, which may not be realistic. 

Plan A is the recruitment that your campus will undertake the week of the survey 
deployment. Based on previous deployments of APPLES, we recommend that the primary 
means of recruiting students be an email from an engineering dean or other senior 
engineering administrator at your institution to engineering students. (We will provide 
suggested text.) We also will be sending you posters tailored to your campus’ APPLES 
participation (please specify how many posters you would like on your Recruitment Plan). 
We have found that posters increase general awareness of APPLES on campus among 
both students and faculty. We have also found that advertising in student newspapers does 
not significantly increase student participation. 

The Survey and Dally Updates; The Orchard APPLE survey will turn “on” at 12:01 am on 
Monday, January 28, 2008, and will turn “off” at 1 1 :59 pm on Friday, February 1 , 2008 
(Pacific Standard Time). During the APPLES deployment on your campus, a member of the 
APPLES team will send you a daily report of survey response rates from your campus by 
strata, noting those strata that have been fulfilled. You will receive the report between 3 and 
6 pm (PST). 

Plan B is the strategic recruitment you may need to undertake during the week of survey 
deployment if your campus' responses are falling short of strata targets. Based on the 
numbers in your daily reports, you (the campus coordinator) will determine if you need to 
implement parts of Plan B. For example, if by February 30 (Wed) only 6 women have filled 
out the survey (and 10 are needed from your campus) - you may decide to send an email to 
the SWE members first thing Thursday (January 31 ) morning. We urge you to be strategic 
in your Plan B recruitment - for example, we have found that targeting specific groups with 
low responses is much more effective than repeating mass emails to all students. 

In rare cases, we and you may choose to extend the deployment period on your campus. 
Our experience with previous deployments indicates that most students take the survey 
immediately following the email announcement or reminder. (Thus, extending the survey 
doesn't buy us much.) 

“Non-persisters” are students who initially intended to study engineering when they arrived 
at university, but later decided to pursue a non-engineering major. We have found the best 
way to recruit non-persister students is to send an email announcement of APPLES to non- 
engineering technical departments at your institution. From your Institutional Overview 
Checklist, these departments are: 

At Orchard: information systems, mathematics, mathematics/computer science, 
chemistry, and bioiogy 


Incentives of $4 paid through PayPal will be offered to all APPLES subjects. The APPLES 
team will handle all aspects of administering incentives to subjects. We reserve the right to 
temporarily close your institution’s survey if we detect widespread fraud (individuals 
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repeatedly claiming the incentive). Should this happen, we will immediately notify you and 
have the survey offline only for the time required to address the problem. 

What’s next? After we receive your institution’s recruitment plan, your APPLES liaison will 
seek to schedule a phone meeting with you (late November/early December). She will 
answer any questions you have and give you a more detailed rundown of what to expect 
during deployment week. 

While this is the last “assignment” from us, there are still some “to do’s” we recommend you 
consider: 

• Determine who has access to the email distribution lists you will need to recruit the 
different strata, and contacting him/her in advance. This may be a bit more challenging, 
for example, with freshmen who intend to study engineering but have not yet declared 
the major. 

• Decide who will hang the APPLES posters, and the most appropriate places given your 
strata targets (and various institutional rules about hanging posters). 

• Come up with a schedule for the week of deployment to help you be sure you hit your 
target responses. In our experience, it is difficult to recruit more than 30% of any 
population (even with the incentive) - so coordinators at smaller schools may need to 
spend more time on strategic recruitment (Plan B) during deployment. We estimate that 
coordinators spend 2-10 hours on APPLES related activities the week leading up to and 
the week of APPLES deployment. 

If you have any questions at all, please don’t hesitate to contact your APPLES liaison. 
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underQraduatc 


A 





take a lO-minute survey 


earn $4 

(through PayPfll) 


The APPLE (Academic Pathways of People Learning Engineering ) survey is trying 
to learn about students’ experience in engineering to improve the educational process. 
That is why it is important to hear from you! It is part of a study funded by the National 
Science Foundation and supported by your institution’s School of Engineering. 


SCHOOL 

EMBLEM 


Appendix 2-D 

Sample APPLES Recruitment Email (to students in the school of engineering) 


Subject: Please take 10 minutes for “APPLES” 

Dear Student, 

[Orchard] is taking the lead in a large, national survey, APPLES, of undergraduate students who 
intended to study engineering when they entered university. We want to learn more about the 
experiences and motivations of students like you. APPLES stands for the Academic Pathways of 
People Learning Engineering Survey. I ask your assistance in taking part in this study hy completing a 

10-15 minute online survey from [February 11-15]. 

Who we are interested in? 

• Undergraduates (18 years of age and older) who are majoring, or ever considered majoring in 
engineering (declared or not) 

Why is this important? 

• Not enough information is available about the experiences of students who studied or considered 
studying engineering - and we want to change this. We hope the information we learn from you 
will lead to improvements in the way engineering is taught in the United States. 

What is involved? 

• A 10-15 minute survey online 

Where do I find it? 

• orchard.applesurvey.org 

When should / take it? 

• The survey is only open from Monday (Feb 11) through Friday (Feb 15). 

And? 

• You will receive $4 through PayPal for your participation, but more importantly you will have 
made a contribution to research and to the education of future generations . 

• All the data we collect are confidential. 

Please take part - and please encourage your friends to take part. 

If you would like more information on the survey, please contact [Orchard’s APPLES coordinator name 
and email]. 

Thank you. 


[Dean/Senior Administrator] 

[Title of Senior Administrator] 
[Institution] School of Engineering 
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Appendix 2-E 

APPLES Recruitment Emaii for Non-persisters (to students NOT in the schooi of 
engineering) 


Subject: Interested in engineering before economics? 

Dear Student, 

Did you declare a major in economics after thinking you were going to study engineering? 

Orchard is taking the lead in a large, national survey, APPLES, of undergraduate students who intended 
to study engineering when they entered university. We want to learn more about the experiences and 
motivations of students like you. APPLES stands for the Academic Pathways of People Learning 
Engineering Survey. I ask your assistance in taking part in this study by completing a 10-15 minute 
online survey from February 11 to February 15. 

Who we are interested in? 

• Undergraduates (18 years of age and older) who ever considered majoring in engineering (declared 
or not) 

Why is this important? 

• Not enough information is available about the experiences of students who studied or considered 
studying engineering - and we want to change this. 

What is involved? 

• A 10-15 minute survey online 

Where do I find it? 

• orchard.applesurvey.org 

When should / take it? 

• The survey is only open from February 11 (Monday) through February 15 (Friday). 

And? 

• You will receive $4 through PayPal for your participation, but more importantly you will have 
made a contribution to research and to the education of future generations . 

• All the data we collect are confidential. 

Please take part - and please encourage your friends to take part. 

If you would like more information on the survey, please contact [Orchard’s APPLES coordinator name 
and email]. 

Thank you, 

[Dean/Senior Administrator] 

[Title of Senior Administrator] 

[Institution] 
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ACADEMIC PATHWAYS STUDY 
INTERVIEW PROTOCOL (EXAMPLE) 
TECHNICAL PUBLIC INSTITUTION/URBAN PRIVATE 
UNIVERSITY/ SUBURBAN PRIVATE UNIVERSITY/ 
LARGE PUBLIC UNIVERSITY 


PART I. 

Instructions 

Good morning ( afternoon ). My name is . Thank you for coming. This interview 

involves two parts. The first part is a survey, in which I will ask you about your 
experiences as a student at this university. The purpose is to get your perceptions of your 
experiences inside and outside of the classroom. There are no right or wrong or 
desirable or undesirable answers. I would like you to feel comfortable with saying what 
you really think and how you really feel. The second part is a short pencil-and-paper 
task, and I will give you specific instructions for completing that task once we have 
finished with the survey. 

Tape Recorder Instructions 

If it is okay with you, I will be tape-recording our conversation. The purpose of this is so 
that I can get all the details but at the same time be able to carry on an attentive 
conversation with you. I assure you that all your comments will remain confidential. I 
will be compiling a report which will contain all students’ comments without any 
reference to individuals. 

Preamble/Consent Form Instructions 

Before we get started, please take a few minutes to read this preamble (read and sign this 
consent form). (Hand R consent form/preamble.) (After R returns preamble/consent 
form, turn tape recorder on.) 


Ql. What is your major? 

Q2. What year did you graduate from high school? 

Q3. Did you participate in a Freshman Summer Bridge Program the summer after you 
graduated from high school? 



Q3a. What were the most helpful aspects of that program? 
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Q4. Are you a member of any engineering student organizations on campus? 


•f 



Skip to Q5 


Q4a. What are those organizations? 


Q4b. What are the most helpful aspects of that/those organization(s)? 


Q5. In your own words, would you please define engineering? 


Q6. Are there particular skills that you would say are important for an engineer to have? 

Skip to Q7 



Q6a. (IF NOT ALREADY ANSWERED) What are those skills? 


Q6b. Of the skills that you mentioned, which ones do you possess? 


Q6c. Please tell me about how you developed your skill(s)? 
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Q7. Have you had any experiences inside or outside of your classes that have enabled 
you to be creative? 



Q7a. (IF NOT ALREADY ANSWERED) Please describe those experiences. 


Q8. Have you had any experiences inside or outside of your classes that have prevented 
you from being creative? 


Yes 


No 


>-Skip to Q9 


Q8a. (IE NOT ALREADY ANSWERED) Please describe those experiences. 
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Q9. Have you had any experiences inside or outside of your classes that have enabled 
you to solve problems? 



Q9a. (IF NOT ALREADY ANSWERED) Please describe those experiences. 


QIO. Have you had any experiences inside or outside of your classes that have prevented 
you from solving problems? 


Yes 


No 


■►Skip to Qll 


QlOa. (IE NOT ALREADY ANSWERED) Please describe those experiences. 
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Q1 1 . Have you had any experiences inside or outside of your classes that have enabled 
you to develop general engineering knowledge? 



Q1 la. (IF NOT ALREADY ANSWERED) Please describe those experiences. 


Q12. Have you had any experiences inside or outside of your classes that have prevented 
you from developing general engineering knowledge? 

Skip to Q13 

Q12a. (IE NOT ALREADY ANSWERED) Please describe those experiences. 
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Q13. Where do you see evidence of your engineering aptitude? 



Q14. On a scale from 0 - 10, (where 0 = not confident at all and 10 = extremely 
confident), how confident are you in your math ability? 

Q14a. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK.) 


Q15. On a scale from 0-10, (where 0 = not confident at all and 10 = extremely 
confident), how confident are you in your science ability? 

Q15a. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK.) 


Q16. On a scale from 0-10, (where 0 = not confident at all and 10 = extremely 
confident), how confident are you in your design ability? 

Q16a. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK.) 
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Q17. Are there any aspects of engineering that you particularly like? 


Yes 


No 


-►Skip to Q1 8 


Q17a. (IF NOT ALREADY ANSWERED) What are some of those? 


Q18. Are there any aspects of engineering that you particularly disl ik e? 



Q18a. (IE NOT AEREADY ANSWERED) What are some of those? 


Q19. In general, how do you feel about engineers? 

Q19a. (IE NOT AEREADY ANSWERED) And why? 


Q20. How do you believe members of other professions feel toward engineers? 
Q20a. (IE NOT AEREADY ANSWERED) And why? 
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Q21. Are any of your family members or close acquaintances working engineers? 

to Q22 

Q21a. (IF NOT ALREADY ANSWERED) Who? 

Q21b. Did their experiences influence your decision to become an 
engineer? 

to Q22 

Q21c. (IE NOT AEREADY ANSWERED) How? 


Q22. How important is being an engineering student to how you feel about yourself? 
Q22a. (IE NOT AEREADY ANSWERED) And why? 

Q23. How committed are you to pursuing an engineering major? 

Q23a. (IE NOT AEREADY ANSWERED) And why? 


Q24. What do you see yourself doing after graduation? 
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Q25. Are there any aspects of your education at this institution that you find particularly 
difficult in achieving your academic goals? 


▼ 


No 


♦^Skip to Q26 


Q25a. (IF NOT ALREADY ANSWERED) Please tell me about those 
difficulties. 


Q25b. How do you deal with those difficulties? 


Q26. Are there any aspects about being an engineering major at this institution that you 
find particularly difficult in achieving your academic goals? 



’’Skip to Q27 


Q26a. (IE NOT AEREADY ANSWERED) Please tell me about those 
difficulties. 


Q26b. How do you deal with those difficulties? 
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Q27. Are there any aspects of your education at this institution that you find particularly 
helpful in achieving your academic goals? 



"Skip to Q28 


Q27a. (IF NOT ALREADY ANSWERED) Please tell me about those 
helpful aspects. 


Q28. Are there any aspects of being an engineering major at this institution that you find 
particularly helpful in achieving your academic goals? 



"Skip to SECTION II 


Q28a. (IE NOT AEREADY ANSWERED) Please tell me about those 
helpful aspects. 
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SECTION II. 

Performance Task (Used in Years I and 3) 

Instructions: 

At this time, I'd like to ask you to work on a short activity. While I hope that it is a fun 
activity for you, I would also like you to give it your best effort. You have up to ten 
minutes to work on it. Please let me know if you are done before that. Do you have any 
questions? OK, here is the activity — I’m going to read it with you, out loud (CHECK 
START TIME ON AUDIO RECORDER AND INDICATE IT IN THE BOX BEEOW. 
THEN HAND R THE PEREORMANCE TASK EORM.). 


Start time: 


RECORD TIME HERE 


PTQI: (NEXT READ THE EOEEOWING AEOUD TO R) Over the summer the 

Midwest experienced massive flooding of the Mississippi River. What factors 
would you take into account in designing a retaining wall system for the 
Mississippi? 

Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): Okay, it’s 
been 10 minutes now, please stop. 


End time: 


RECORD TIME HERE 


PTQ2: What questions came to your mind as you were brainstorming your list? 


(THE QUESTIONS NEED TO BE EUEEY EORMUEATED. IE R OEEERS A 
ERAGMENT AS A QUESTION, INSTRUCT HIM/HER TO CEARIEY HOW HE/SHE 
USED IT IN A QUESTION AND TO STATE THE EUEE QUESTION. IT IS OKAY 
EOR R TO BROWSE THROUGH THE EIST OE EACTORS HE/SHE HAS WRITTEN, 
BUT THERE IS NO NEED TO SUGGEST THIS IN YOUR INSTRUCTIONS. R 
SHOUED BE ABEE TO PROVIDE 5-10 QUESTIONS IN 2-3 MINUTES.) 

Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we 
want to make sure that you know that there are many right answers. We ’ve used it to 
collect information from engineering students across the nation to understand the types 
of things students think about. 
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Over the summer the Midwest experienced massive flooding of the Mississippi 
River. What factors would you take into account in designing a retaining wall 
system for the Mississippi? 
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SECTION II. 

Performance Task (Used in Years 2 and 4) 


Instructions: 

At this time, I'd like to ask you to work on a short activity. This is the kind of activity that 
has many different kinds of answers. We would like you to give it your best effort. You 
have up to fifteen minutes to work on it. I will let you know when there is five minutes left, 
so you have an idea about how much time has passed. Please let me know if you are 
done before the fifteen minutes is up. Do you have any questions? OK, here is the 
activity. (CHECK START TIME ON AUDIO RECORDER AND INDICATE IT IN 
THE BOX BEEOW. THEN HAND R THE PEREORMANCE TASK EORM.). 


Start time: 


RECORD TIME HERE 


PTQI. (AEEOW THE STUDENT TO READ AND SOEVE THE PROBEEM ON 
THEIR OWN) 


Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): 
Okay, it’s been 10 minutes now; you have 5 more minutes to solve the problem. 
Okay, it’s been 15 minutes now, please stop. 


End time: 


RECORD TIME HERE 


PTQ2. What questions came to your mind as you were solving the problem? Please voice 
your thoughts in the form of questions as if you are playing Jeopardy. 


(THE QUESTIONS MUST BE EUEEY EORMUEATED. IE R OEEERS A 
ERAGMENT AS A QUESTION, REMIND R TO SPEAK AS IE HE/SHE IS PEAYING 
JEOPARDY, AND ASK R TO CEARIEY HOW HE/SHE USED THE ERAGMENT IN 
A QUESTION. IE R IS NOT EAMIEIAR WITH JEOPARDY, TEEE R THAT IT IS 
OKAY, AND THAT AEE HE/SHE NEEDS TO DO IS TO RESPOND IN QUESTIONS 
ONEY. IT IS OKAY EOR R TO BROWSE THROUGH THE EIST OE SOEUTIONS 
HE/SHE HAS WRITTEN, BUT THERE IS NO NEED TO SUGGEST THIS IN YOUR 
INSTRUCTIONS. IE R OEEERS 2 QUESTIONS OR EESS, PROMPT HIM/HER 
AGAIN. R SHOUED BE ABEE TO PROVIDE 5-10 QUESTIONS IN 2-3 MINUTES.) 


3 A- 14 


APS Research Processes and Procedures 

January 2009 





PTQ3. To what extent do you feel this is an engineering problem? 


PTQ3a. (IF NOT ALREADY ANSWERED) And why? 


PTQ4. What knowledge and skills helped you solve the problem? 


PTQ5. Where did you develop your knowledge and skills to solve the problem? 


PTQ5a. (IE NOT AEREADY ANSWERED) Please describe those experiences 
in more detail. 


Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we 
want to make sure that you know that there are many right answers. We ’ve used it to 
collect information from engineering students across the nation to understand the types 
of things students think about. 
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Respondent ID: 

As an engineer, you have been asked to solve a problem on the State University campus. 
Just l ik e campuses across the country, the State University campus is often overcrowded 
with pedestrians crossing the streets. 


One busy intersection on campus is the crossing of Fifth Ave. in front of the bookstore. 
Dangers at this intersection include heavy traffic and busses which run against the 
general traffic flow (see diagram below). The University would like to design a cost 
effective method for students to cross Fifth Ave. which would reduce the possibility of 
accidents at this intersection. You have been assigned to design a solution to this 
problem for presentation to the University Traffic Committee. 


One lane 
for buses 
travelingx 
south 




▼ 





Book 

Store 





Parking 

Lot 


In the process of designing your solution you have been asked to respond to the set of 
questions on the following pages. The interviewer has more paper if you need it. 
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1 - What is the problem as you see it? 


2 - List potential solution(s) for this problem. 


3 - From your list in Question 2, choose the potential solution you think is best and 
provide a detailed evaluation of your solution. 


4 - What kinds of additional information would help you solve this problem? 
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SECTION IV. 

Debriefing 

(READ ALL OL THE LOLLOWING ALOUD TO R.) 

Thank you very much for coming this morning (afternoon). Your time is very much 
appreciated and your comments have been very helpful. 

The purpose of this interview is to better understand students’ perceptions of their 
experiences inside and outside of the classroom. We are interested in your opinions and 
your reactions. In no way is this interview designed to individually evaluate a person’s 
abilities. The task is not diagnostic, nor can it provide a measure of the “quality” of your 
performance. Your only requirement was to do the best job that you could. 

The results of this research will provide useful information to engineering educators, in 
helping them to structure educational programs that students consider to be most effective 
and ideal in helping them through college. 

You will be kept anonymous during all phases of this study including any experimental 
writings, published or not. Procedures for maintaining confidentiality are as follows: (1) 
individual participants’ results will be pooled with group results; and (2) participants should 
not place any identifying information on data collection instruments. (Such identifiers 
include name, social security number, student identification number, specific birth data, 
telephone number, address, etc.) 

DQl. Is there any other information regarding your experience that you think would be 
useful for me to know? 



Again, thank you for participating. (TURN TAPE-RECORDER OLE.) 
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SECTION V. 

Interviewer Reelection 


Instructions: 

After the respondent leaves the room, please take a couple of minutes to indicate your 
reactions and observations about the interview. An electronic copy of this form has been 
provided. Feel free to use this hard copy for your own notes, but please submit the 
electronic copy for official use. 


Your name 
(the interviewer): 


Your race: 


Your gender: 


Your age: 



Respondent ID No.: 


Date of Interview: 


Please describe the 
respondent’s attitude 
toward you and the 
interview: 


Please describe any 
unusual 

circumstances and/ 
or events that had 
any bearing on the 
interview such as 
interruptions, 
language difficulty, 
etc.: 


Please describe 
anything else that 
happened during the 
interview that has 
any bearings on the 
study’s objectives: 


Additional 

comments: 
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SECTION VI. 

Probes for formal interview 


Interviewer’s Probe 


Repeat Question 

Anything else? 

Any others? 

How do you mean? 

Could you tell me more about your 

thinking on that? 

Would you tell me what you have 

in mind? 

What do you mean? 

Why do you feel that way? 

Which would he closer to the way 
you fell? 


Abbreviation 

RQ 

AE or Else? 
AO? 

How mean? 

Tell more 

What in mind? 
What mean? 
Why? 

Which closer? 
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SECTION VII. 

Potential questions R may ask (and appropriate answers): 


1 . What is a factor? 

You just need to “ List the things that you would take into account in designing a 
retaining wall system for the Mississippi. ” 

2. What is a retaining wall system? 

A system that keeps water inside the river. 

3. What is massive flooding? 

When a lot of water does not stay inside the river. 

4. Is a wall necessary? / Does it need to be a wall? / Why a retaining wall? 

You can interpret the question however you like. 

5. Who asked to have the wall built? 

I don ’t have any information on that. 

6. Midwest of what? 

The Midwest of the United States. 

7. The whole Mississippi, or just part of it? 

You can do what you like. 

8. List vs. pictures 

You can do what you like. 

9. Quantity vs. quality of responses. 

You can do what you like. 

10. What do they use now to control the flooding? 

I don ’t have any information on that. 

11. Does this mean a retaining wall like on the sides of the river? 

Yes, it is a system that keeps water inside the river. 

12. Is that what you want(ed)? 

That’s great! Thanks! 
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Appendix 3-B 

APS Engineering Task Protocois and ETD Data Sets 


Problem-Scoping Task Protocol 2004 1-2 

Performance Task 2005 T3 

Performance Task 2006 T8 

Performance Task 2007 1-l 1 

ETD Data Sets T14 
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Problem-Scoping Task Protocol 
( 2004 ) 


Instructions: 

At this time, I'd like to ask you to work on a short activity. While I hope that it is a fun 
activity for you, I would also like you to give it your best effort. You have up to ten 
minutes to work on it. Please let me know if you are done before that. Do you have any 
questions? OK, here is the activity — I’m going to read it with you, out loud (CHECK 
START TIME ON AUDIO RECORDER AND INDICATE IT IN THE BOX BEEOW. 
THEN HAND R THE PROBEEM-SCOPING TASK EORM.). 

Start time: 


PTQI: (NEXT READ THE EOEEOWING AEOUD TO R) Over the summer the 

Midwest experienced massive flooding of the Mississippi River. What factors 
would you take into account in designing a retaining wall system for the 
Mississippi? 

Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): Okay, it’s 
been 10 minutes now, please stop. 

End time: 


PTQ2: What questions came to your mind as you were brainstorming your list? 


(THE QUESTIONS NEED TO BE EUEEY EORMUEATED. IE R OEEERS A 
ERAGMENT AS A QUESTION, INSTRUCT HIM/HER TO CEARIEY HOW HE/SHE 
USED IT IN A QUESTION AND TO STATE THE EUEE QUESTION. IT IS OKAY 
EOR R TO BROWSE THROUGH THE EIST OE EACTORS HE/SHE HAS WRITTEN, 
BUT THERE IS NO NEED TO SUGGEST THIS IN YOUR INSTRUCTIONS. R 
SHOUED BE ABEE TO PROVIDE 5-10 QUESTIONS IN 2-3 MINUTES.) 

Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we 
want to make sure that you know that there are many right answers. We ’ve used it to 
collect information from engineering students across the nation to understand the types 
of things students think about. 
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Performance Task (2005) 


Instructions: 

At this time, I'd like to ask you to work on a short activity. This is the kind of activity that 
has many different kinds of answers. We would like you to give it your best effort. You 
have up to fifteen minutes to work on it. I will let you know when there is five minutes left, 
so you have an idea about how much time has passed. Please let me know if you are 
done before the fifteen minutes is up. Do you have any questions? OK, here is the 
activity. (CHECK START TIME ON AUDIO RECORDER AND INDICATE IT IN 
THE BOX BEEOW. THEN HAND R THE PEREORMANCE TASK EORM.). 


Start time: 


RECORD TIME HERE 


PTQI. (AEEOW THE STUDENT TO READ AND SOEVE THE PROBEEM ON 
THEIR OWN) 
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Respondent ID: 

As an engineer, you have been asked to solve a problem on the State University campus. 
Just like campuses across the country, the State University campus is often overcrowded 
with pedestrians crossing the streets. 


One busy intersection on campus is the crossing of Fifth Ave. in front of the bookstore. 
Dangers at this intersection include heavy traffic and busses which run against the 
general traffic flow (see diagram below). The University would like to design a cost 
effective method for students to cross Fifth Ave. which would reduce the possibility of 
accidents at this intersection. You have been assigned to design a solution to this 
problem for presentation to the University Traffic Committee. 


One lane 
for buses 









Parking 

Lot 


In the process of designing your solution you have been asked to respond to the set of 
questions on the following pages. The interviewer has more paper if you need it. 
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1 - What is the problem as you see it? 


2 - List potential solution(s) for this problem. 


3 - From your list in Question 2, choose the potential solution you think is best and 
provide a detailed evaluation of your solution. 


4 - What kinds of additional information would help you solve this problem? 
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Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): 
Okay, it’s been 10 minutes now; you have 5 more minutes to solve the problem. 
Okay, it’s been 15 minutes now, please stop. 


End time: 


RECORD TIME HERE 


PTQ2. What questions came to your mind as you were solving the problem? Please voice 
your thoughts in the form of questions as if you are playing Jeopardy. 


(THE QUESTIONS MUST BE EUEEY EORMUEATED. IE R OEEERS A 
ERAGMENT AS A QUESTION, REMIND R TO SPEAK AS IE HE/SHE IS PEAYING 
JEOPARDY, AND ASK R TO CEARIEY HOW HE/SHE USED THE ERAGMENT IN 
A QUESTION. IE R IS NOT EAMIEIAR WITH JEOPARDY, TEEE R THAT IT IS 
OKAY, AND THAT AEE HE/SHE NEEDS TO DO IS TO RESPOND IN QUESTIONS 
ONEY. IT IS OKAY EOR R TO BROWSE THROUGH THE EIST OE SOEUTIONS 
HE/SHE HAS WRITTEN, BUT THERE IS NO NEED TO SUGGEST THIS IN YOUR 
INSTRUCTIONS. IE R OEEERS 2 QUESTIONS OR EESS, PROMPT HIM/HER 
AGAIN. R SHOUED BE ABEE TO PROVIDE 5-10 QUESTIONS IN 2-3 MINUTES.) 


PTQ3. To what extent do you feel this is an engineering problem? 


PTQ3a. (IE NOT AEREADY ANSWERED) And why? 


PTQ4. What knowledge and skills helped you solve the problem? 
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PTQ5. Where did you develop your knowledge and skills to solve the problem? 


PTQ5a. (IF NOT ALREADY ANSWERED) Please describe those experiences 
in more detail. 


PTQ6. Did you feel confident in your ability to complete this task? 
PTQ6a. (IP NOT AEREADY ANSWERED) And why? 


Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we 
want to make sure that you know that there are many right answers. We’ve used it to 
collect information from engineering students across the nation to understand the types 
of things students think about. 
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Performance Task (2006) 


Transition for Ethnographic Interviews: 

The last part of today’s session has a different format from the interview we just 
completed. I will be reading instructions and questions from a script. 


Instructions: 

At this time, I'd like to ask you to work on a short activity. While I hope that it is a fun 
activity for you, I would also like you to give it your best effort. You have up to ten 
minutes to work on it. Please let me know if you are done before that. Do you have any 
questions? 


OK, here is the activity — I’m going to read it with you, out loud. You might remember 
this activity if you were asked to do it two years ago as part of this study. (CHECK 


START TIME ON AUDIO RECORDER AND INDICATE IT IN THE BOX BEEOW. 
THEN HAND R THE PEREORMANCE TASK EORM.). 


Start time: 


HH MM SS 


PTQl : (NEXT READ THE EOEEOWING AEOUD TO R) Over the summer the 

Midwest experienced massive flooding of the Mississippi River. What factors 
would you take into account in designing a retaining wall system for the 
Mississippi? 


Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): Okay, it’s 
been 10 minutes now, please stop. 

End time: 


PTQl: What questions came to your mind as you were brainstorming your list? 


(THE QUESTIONS MUST BE EUEEY EORMUEATED. IE R OEEERS A 
ERAGMENT AS A QUESTION, REMIND R TO SPEAK AS IE HE/SHE IS PEAYING 
JEOPARDY, AND ASK R TO CEARIEY HOW HE/SHE USED THE ERAGMENT IN 
A QUESTION. IE R IS NOT EAMIEIAR WITH JEOPARDY, TEEE R THAT IT IS 
OKAY, AND THAT AEE HE/SHE NEEDS TO DO IS TO RESPOND IN QUESTIONS 
ONEY. IT IS OKAY EOR R TO BROWSE THROUGH THE EIST OE SOEUTIONS 
HE/SHE HAS WRITTEN, BUT THERE IS NO NEED TO SUGGEST THIS IN YOUR 
INSTRUCTIONS. IE R OEEERS 2 QUESTIONS OR EESS, PROMPT HIM/HER 
AGAIN. R SHOUED BE ABEE TO PROVIDE 5-10 QUESTIONS IN 2-3 MINUTES.) 
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(IF YOU HAVE A YEAR 1 MIDWEST EEOODS WRITTEN RESPONSE EOR R, 
GIVE IT TO THEM AND PROCEED TO PTQ3. IE NOT, SKIP AHEAD TO PTQ5, 
WHICH IS MARKED WITH AT.) 

Here’s a copy of your list of factors from back in 2004, when you first did the Mississippi 
flooding activity. 

PTQ3: Take a look at both the response you just wrote today and your response from 

two years ago. What similarities and differences do you notice between the two 
responses? 


PTQ4: You ’ve told me a little about how your responses are similar or different. How 

about how you came up with them ? Consider how you thought about the 
activity and how you came up with the factors you wrote down, both today and 
two years ago. What similarities and differences do you notice? 

(IT’S EINE IE R AEREADY BEGAN COMPARING THOUGHT PROCESSES (VS. 
COMPARING RESPONSES) IN ANSWERING THE PREVIOUS QUESTION (PTQ3). 
ASK THEM TO CONTINUE, E.G., “Do you notice any other similarities or differences 
in the way you came up with your response?”) 


T 

PTQ5: Have you had any past experiences that helped you do the written activity? 

(IE SO, ASK R TO DESCRIBE THE EXPERIENCES.) 


PTQ6: Have you had any educational experiences that helped you do this activity? 

(IE SO, ASK R TO DESCRIBE THE EXPERIENCES. R MIGHT HAVE AEREADY 
DISCUSSED EDUCATIONAE EXPERIENCES IN RESPONDING TO PTQ5. IP SO, 
ASK THEM TO CONTINUE, E.G., “Are there any other educational experiences that 
helped you do the activity?”, OR ASK EOR ADDITIONAE DETAIES ABOUT THE 
EDUCATIONAE EXPERIENCES, IP TIME PERMITS.) 
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(IF YOU DID HAVE A YEAR 1 MIDWEST EEOODS WRITTEN RESPONSE EOR R, 
SKIP AHEAD TO THE CEOSING TRANSITION BEEOW, MARKED WITH A ■.) 


Transition for students who did not do the activity in Year 1: 

Now, I’d like to ask you some questions about a recent natural disaster in the U.S. 

PTQ7: How familiar are you with Hurricane Katrina and the flooding in New Orleans? 

Could you tell me what you know about these events? 


PTQ8: Did what you know about these events affect how you approached the 

Mississippi flooding activity today? 

(IE SO, ASK R TO DESCRIBE HOW THEIR KNOWEEDGE AEEECTED THEIR 
APPROACH TO THE ACTIVITY.) 


Closing transition eor all students: 

(COEEECT BOTH 2004 AND TODAY’S RESPONSE PAGES.) Great! Thank you. We 
hope that you ’ve enjoyed this activity and we want to make sure that you know that there 
are many right answers. We’ve used it to collect information from engineering students 
across the nation to understand the types of things students think about. 
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Performance Task 2007 


Transition from Ethnographic Interview: 

The last part of today’s session has a different format from the interview we just 
completed. I will be reading instructions and questions from a script. 

Instructions: 

At this time, I'd like to ask you to work on a short activity. This is the kind of activity that 
has many different kinds of answers. We would like you to give it your best effort. You 
have up to fifteen minutes to work on it. I will let you know when there is five minutes left, 
so you have an idea about how much time has passed. Please let me know if you are 
done before the fifteen minutes is up. Do you have any questions? OK, here is the 
activity. You might remember this activity if you were asked to do it two years ago as 
part of this study. (CHECK START TIME ON AUDIO RECORDER AND INDICATE 
IT IN THE BOX BEEOW. THEN HAND R THE EOUR-PAGE PEREORMANCE 
TASK PACKET.) 


Start time: 


HH MM SS 


PTQl. (AEEOW THE STUDENT TO READ AND SOEVE THE PROBEEM ON 
THEIR OWN) 


Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): 
Okay, it’s been 10 minutes now; you have 5 more minutes to solve the problem. 
Okay, it’s been 15 minutes now, please stop. 


End time: 


HH MM SS 


(IE YOU HAVE A YEAR 2 STREET CROSSING WRITTEN RESPONSE EOR R, 
GIVE IT TO THEM AND PROCEED TO PTQ2 ON THE NEXT PAGE. IE NOT, SKIP 
AHEAD TO PTQ4, WHICH IS MARKED WITH AT.) 
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Here’s a copy of your responses from back in 2005, when you first did the street crossing 
activity. 

PTQ2: Take a look at both the responses you just wrote today and your responses from 

two years ago. What similarities and differences do you notice between the two 
sets of responses? 


PTQ3: You ’ve told me a little about how your responses are similar or different. How 

about how you came up with them? Consider how you thought about the 
activity and how you came up with the responses you wrote down, both today 
and two years ago. What similarities and differences do you notice? 

(IT’S FINE IF R ALREADY BEGAN COMPARING THOUGHT PROCESSES (VS. 
COMPARING RESPONSES) IN ANSWERING THE PREVIOUS QUESTION (PTQ2). 
ASK THEM TO CONTINUE, E.G., “Do you notice any other similarities or differences 
in the way you came up with your response?”) 


T 

PTQ4: To what extent do you feel this is an engineering problem? 


PTQ4a: (IF NOT ALREADY ANSWERED) And why? 


PTQ5: What knowledge and skills helped you solve the problem? 
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PTQ6: Where did you develop your knowledge and skills to solve the problem? 


PTQ6a: (IF NOT ALREADY ANSWERED) Please describe those experiences 
in more detail. 


PTQ7: Are there any everyday situations from your life that remind you of the situation 
described in the problem? 


Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we 
want to make sure that you know that there are many right answers. We ’ve used it to 
collect information from engineering students across the nation to understand the types 
of things students think about. 
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APS Engineering Thinking & Doing Group data set reference (2008.04.24) 



data set 

method 


Year 





1 

2 

3 

4 

la 

Engineering design task: Midwest floods 

engineering 



y 


lb 

Engineering design task: Street crossing 

design task 


y 


y 

2 

Most/least important design activities 

survey 


y 

y 

y 

3 

5 terms describing "engineering", "design" 

survey 



y 

y 

4 

Confidence in, course experience with, and course prep, for design activities 

survey 


y 

y 

y 

5^ 

Skills important for engineering 

structured interview 


y 

y 


6 

Perceived importance of engr. knowledge, skills (technical, professional, 
interpersonal) 

survey 


y 

y 

y 

7a 

Engineering design task: Most/least needed information for playground design 

survey 




y 

7b 

Most/least needed information for typical engineering problem 

survey 



y 


8a 

Most important skills and knowledge for engineering (from ABET, Engr. of 2020 list) 

survey 



y 

y 

8b 

Self assessment of preparation to use engineering skills and knowledge (from ABET, 
Engr. of 2020 list) 

survey 




y 

9 

5 activities engineers do at work 

survey 



y 

y 

10a 

1 mportance of various kinds of context in engineering 

survey 



y 


10b 

Self assessment of preparation to consider various kinds of context in engineering 

survey 




y 

11 

Engineering design task: Five factors important for evaluating silicon chip factory 
location 

survey 




y 

12 

Experience in academic research and professional engineering 

survey 




y 

13 

Overall academic satisfaction 

survey 


y 

y 

y 


^not an ETD data set; included for reference as potential set for joint analysis with ETD data 
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Background Information for Interviewers 


Campus Contacts for the Formal Interviews: 

• List contacts for Technical Public Institution, Urban Private University, Suburban Private 
University, and Large Public University. 

Campus Contacts for the Ethnographic Interviews: 

• List contacts for Technical Public Institution, Urban Private University, Suburban Private 
University, and Large Public University. 

APS Librarian : Librarian [aaaa@yahoo.com] 

CALL APS Database : Database Manager rbbbb@withinc.com1 

Intranets : Intranets Manager [cccc@engr.lpub.edu] 

Performance-Scoping Task : Researcher [dddd@cs.lpub.edu] 

Calendar Scheduling: 

• List contacts for Technical Public Institution, Urban Private University, Suburban Private 
University, and Large Public University. 

Please contact CALL Assistant Director [cccc@engr.lpub.edu] for general assistance with the 
scheduling tool. 


General Help and Information : Please cc and/or call the researcher [phone #] on all questions 
related to the interview protocols, process, etc. so we can keep track of what issues are coming 
up and also direct you to the right persons and resources. 

Interview Descriptions: 

• No students will receive both interview types. 

• All 40 students per campus will do the performance task as part of the interview 

• There will be a total of 48 events organized as follows: 

o 24 formal interviews and performance tasks with the unobserved students 
o 16 ethnographic interviews and performance task with the eight observed students 
and eight unobserved students who were “combo” students during the ’04 
interview year 


Formal Interview Packets: What you need to have in hand before each interview 

For each formal interview, print out a copy of each of the following for each respondent: 

1 . APS Formal Interview Protocol 2006 

2. Your School’s Preamble 

3. Performance Task Sheet 

4. Extra Blank Paper for Performance Task 

5. Interviewer Reflection Sheet 
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The only sheets each respondent actually sees are: 1) the Performance Task Sheet; and 2) the 

Preamble. 

A packet with all of the above has been created with the following names: 

• TPub APS Formal Interview Packet 2006 

• UPri APS Formal Interview Packet 2006 

• SPri APS Formal Interview Packet 2006 

• LPub APS Formal Interview Packet 2006 

Formal Interview Reminders: 

• Follow the protocol. 

• Ask complete questions. 

• Remember to ask “And why” for certain questions. 

• Do not skip/omit questions. If respondent asks you to skip a question, be sure to return to the 
question before administering the Performance Task. 

• Do not initiate personal conversations. 

• Use appropriate probes when respondents do not answer the question. (See Probes for 
Formal Interviews and Usefulness Document for question’s objectives.) However, do not ask 
new questions. 

• Do not offer suggestions as to how the respondent should answer the question. (For example, 
do not say that other students have been answering the question such-and-such a way.) 

• Make sure you complete the Interviewer Reflection form immediately after the interview is 
completed and preferably, before the next respondent arrives. 

• Remember, this is NOT an ethnographic interview. 

• Do not offer advice during the interviews. This is not a counseling session. (If a major issue 
arises, make a note of it in the Interviewer Reflection Document.) 


Ethnographic Interview Packets: What you need to have in hand before each interview 

For each ethnographic interview, print out a copy of each of the following for each respondent: 

1 . APS Ethnographic Interview Protocol 2006 

2. Your School’s Preamble 

3. Performance Task Sheet 

4. Extra Blank Paper for Performance Task 

The only sheets each respondent actually sees are: 1) the Performance Task Sheet; and 2) the 
Preamble. 

A packet with all of the above has been created with the following names: 

• TPub APS Ethnographic Interview Packet 2006 

• UPri APS Ethnographic Interview Packet 2006 

• SPri APS Ethnographic Interview Packet 2006 

• EPub APS Ethnographic Interview Packet 2006 
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Before each respondent arrives: 

• Check the batteries and the recorder. Make sure there is enough space on the memory card. 

• Check the microphone battery. 

• Make sure your interview room does not have any distractions (turn off phones, close door, 
etc.). 


INTERVIEW EMAIL NOTIFICATION 
Formal Interview 

Email for a Person doing the Scheduling 

Greetings from the Center for the Advancement of Engineering Education research team! 

We will be conducting the APS interviews from (START DATE) through (END DATE). We 
anticipate that it will take approximately one hour to complete the interview. However, 
recognizing that interview length naturally will vary for each individual (for example some 
interviews may take a few as 45 minutes, while others may take up to an hour and 15 minutes), 
we are asking all participants to sign up for Wi hour slots to avoid scheduling overlaps. 

To arrange your interview, please call (NAME) at (Number) or report to (Building Name, Room 

# ) to select a date and time that is convenient to your schedule. The interviews will be 

held in the (Building Name, Room # ). 

Your continuous participation in the APS is valuable to us. Thank you again for helping us to 
improve the quality of engineering education. 

Sincerely, 

Academic Pathways Study Coordinator 

Center for the Advancement of Engineering Education 


Email for Calendar System Scheduling 

Greetings from the Center for the Advancement of Engineering Education research team! 

We will be conducting the APS interviews from (START DATE) through (END DATE). We 
anticipate that it will take approximately one hour to complete the interview. However, 
recognizing that interview length naturally will vary for each individual (for example some 
interviews may take as few as 45 minutes, while others may take up to an hour and 15 minutes), 
we are asking all participants to sign up for IVi hour slots to avoid scheduling overlaps. 
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To arrange your interview, please log on to our on-line interview scheduling system at: 

(LINK) 

In the “LAST NAME” field, type: #LoginID# 

In the “PASSWORD” field, type: #Password# 

You have until 4/14/2006, 5:00 pm to sign up. You will not be able make any changes on the 
system. If you need to make a change after the 14* due to circumstances beyond your control, 
please contact your interviewer directly at the email address provided by the system. If you have 
difficulty using the scheduling system, please email (NAME) at (EMAIE ADDRESS). 

The interviews will be held at the (Building Name, Room # ). Your interviewer will greet 

you at (location). 

Your continuous participation in the APS is valuable to us. Thank you again for helping us to 
improve the quality of engineering education. 

Sincerely, 

Academic Pathways Study Coordinator 

Center for the Advancement of Engineering Education 


Ethnographic Interview 

Email for a Person doing the Scheduling 

Greetings from the Center for the Advancement of Engineering Education research team! 

We will be conducting the APS interviews from (START DATE) through (END DATE). We 
anticipate that it will take approximately 90 minutes to complete the interview. However, 
recognizing that interview length naturally will vary for each individual, we are asking all 
participants to sign up for two-hour slots to avoid scheduling overlaps. 

To arrange your interview, please call (NAME) at (Number) or report to (Building Name, Room 

# ) to select a date and time that is convenient to your schedule. The interviews will be 

held in the (Building Name, Room # ). 

Your continuous participation in the APS is valuable to us. Thank you again for helping us to 
improve the quality of engineering education. 

Sincerely, 
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Academic Pathways Study Coordinator 

Center for the Advancement of Engineering Education 


Email for Calendar System Scheduling 

Greetings from the Center for the Advancement of Engineering Edueation researeh team! 

We will be conducting the APS interviews from (START DATE) through (END DATE). We 
anticipate that it will take approximately 90 minutes to eomplete the interview. However, 
reeognizing that interview length naturally will vary for eaeh individual, we are asking all 
participants to sign up for two-hour slots to avoid scheduling overlaps. 

To arrange your interview, please log on to our on-line interview scheduling system at: 

(EINK) 

In the “EAST NAME” field, type: #EoginID# 

In the “PASSWORD” field, type: #Password# 

You have until 4/14/2006, 5:00 pm to sign up. You will not be able make any changes on the 
system. If you need to make a ehange after the I4‘^ due to eircumstances beyond your control, 
please contact your interviewer directly at the email address provided by the system. If you have 
difficulty using the seheduling system, please email (NAME) at (EMAIE ADDRESS). 

The interviews will be held at the (Building Name, Room # ). Your interviewer will greet 

you at (loeation). 

Your continuous participation in the APS is valuable to us. Thank you again for helping us to 
improve the quality of engineering education. 

Sincerely, 

Aeademie Pathways Study Coordinator 

Center for the Advaneement of Engineering Education 


3C-8 


APS Research Processes and Procedures 

May 2009 


Preambles 


Technical Public Institution Preamble 


Preamble for Investigative Procedures 
NSF Center for the Advancement of Engineering Education 
Urban Private University 

This is an investigation in the Center for Engineering Education. This study is being conducted by 
a CAEE researcher. The purpose of this study is to gain an in-depth understanding of how 
engineering students explore and utilize the educational opportunities available to them in 
college, plan and carry out their curriculums (successfully or unsuccessfully), and navigate the 
process of becoming engineers. 

The purpose of the individual interview is to obtain a deeper understanding of students’ 
perceptions, expectations, motivations, practices, and experiences in engineering. We are 
interested in your perceptions of your experiences inside and outside of the classroom. In no 
way is this interview designed to individually evaluate a person’s abilities. The interview is not 
diagnostic. Please answer the interview questions to the best of your ability. 

We anticipate minimal psychological risks and personal time inconvenience. Participants will be 
compensated $175.00 per year for participating in a combination of survey, interview, and 
observation methods. Annual payments will be made at the end of spring semester upon 
continuous participation for that school year. 

All of the information that we gather on this project will remain strictly anonymous and will not 
identify you in any way. The interview sessions will be audio taped, and the ethnographic 
sessions will be audio taped, photographed, and videotaped so that we can review them later. 
However, all identifying information will be removed and replaced by code numbers. 
Participant codes will be kept in a logbook, which will be stored in a locked file cabinet separate 
from the actual data information. We will keep all materials in a secure place in our university 
office. 

The participants should be 18 years of age or older and in good health. If you are younger than 18, 
please contact the investigator immediately. 

The Project Researcher may be contacted at [phone #] in the event that you have any questions 
regarding your participation in this project. If you have questions about your rights as a study 
participant, or are dissatisfied at any time with any aspect of this study, you may contact - 
anonymously, if you wish - the Office of Executive Secretary, Urban Private University 
Institutional Review Board at [phone #]. 
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Urban Private University Preamble 

Preamble for Investigative Procedures 
NSF Center for the Advancement of Engineering Education 
Urban Private University 

This is an investigation in the Center for the Advancement of Engineering Education. This study is 
being conducted by a CAEE researcher. The purpose of this study is to gain an in-depth 
understanding of how engineering students explore and utilize the educational opportunities 
available to them in college, plan and carry out their curriculums (successfully or 
unsuccessfully), and navigate the process of becoming engineers. 

The purpose of the individual interview is to obtain a deeper understanding of students’ 
perceptions, expectations, motivations, practices, and experiences in engineering. We are 
interested in your perceptions of your experiences inside and outside of the classroom. In no 
way is this interview designed to individually evaluate a person’s abilities. The interview is not 
diagnostic. Please answer the interview questions to the best of your ability. 

We anticipate minimal psychological risks and personal time inconvenience. Participants will be 
compensated $175.00 per year for participating in a combination of survey, interview, and 
observation methods. Annual payments will be made at the end of spring semester upon 
continuous participation for that school year. 

All of the information that we gather on this project will remain strictly anonymous and will not 
identify you in any way. The interview sessions will be audio taped, and the ethnographic 
sessions will be audio taped, photographed, and videotaped so that we can review them later. 
However, all identifying information will be removed and replaced by code numbers. 
Participant codes will be kept in a logbook, which will be stored in a locked file cabinet separate 
from the actual data information. We will keep all materials in a secure place in our university 
office. 

The participants should be 18 years of age or older and in good health. If you are younger than 18, 
please contact the investigator immediately. 

The Project Researcher may be contacted at [phone #] in the event that you have any questions 
regarding your participation in this project. If you have questions about your rights as a study 
participant, or are dissatisfied at any time with any aspect of this study, you may contact - 
anonymously, if you wish - the Office of Executive Secretary, Urban Private University 
Institutional Review Board at [phone #]. 
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Suburban Private University Preamble 

Preamble for Investigative Procedures 
NSF Center for the Advancement of Engineering Education 
Suburban Private University 

This is an investigation in the Center for Design Research. This study is being conducted by a 
CAEE researcher. The purpose of this study is to gain an in-depth understanding of how 
engineering students explore and utilize the educational opportunities available to them in 
college, plan and carry out their curriculums (successfully or unsuccessfully), and navigate the 
process of becoming engineers. 

The purpose of the individual interview is to obtain a deeper understanding of students’ 
perceptions, expectations, motivations, practices, and experiences in engineering. We are 
interested in your perceptions of your experiences inside and outside of the classroom. In no 
way is this interview designed to individually evaluate a person’s abilities. The interview is not 
diagnostic. Please answer the interview questions to the best of your ability. 

We anticipate minimal psychological risks and personal time inconvenience. Participants will be 
compensated $175.00 per year for participating in a combination of survey, interview, and 
observation methods. Annual payments will be made at the end of spring semester upon 
continuous participation for that school year. 

All of the information that we gather on this project will remain strictly anonymous and will not 
identify you in any way. The interview sessions will be audio taped, and the ethnographic 
sessions will be audio taped, photographed, and videotaped so that we can review them later. 
However, all identifying information will be removed and replaced by code numbers. 
Participant codes will be kept in a logbook, which will be stored in a locked file cabinet separate 
from the actual data information. We will keep all materials in a secure place in our university 
office. 

The participants should be 18 years of age or older and in good health. If you are younger than 18, 
please contact the investigator immediately. 

The Project Researcher, may be contacted at [phone #] in the event that you have any questions 
regarding your participation in this project. If you have questions about your rights as a study 
participant, or are dissatisfied at any time with any aspect of this study, you may contact - 
anonymously, if you wish - the Administrative Panels Office, Suburban Private University 
[contact info] . 
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Large Public University Preamble 

PREAMBLE FOR INVESTIGATIVE PROCEDURES 
ACADEMIC PATHWAYS LONGITUDINAL STUDY 
LARGE PUBLIC UNIVERSITY 

This is an investigation in the College of Education. This study is being conducted by CAEE 
researchers. The purpose of this study is to gain an in-depth understanding of how engineering 
students explore and utilize the educational opportunities available to them in college, plan and 
carry out their curriculums (successfully or unsuccessfully), and navigate the process of 
becoming engineers. 

The purpose of the individual interview is to obtain a deeper understanding of students’ 
perceptions, expectations, motivations, practices, and experiences in engineering. We are 
interested in your perceptions of your experiences inside and outside of the classroom. In no 
way is this interview designed to individually evaluate a person’s abilities. The interview is not 
diagnostic. Please answer the interview questions to the best of your ability. 

We anticipate minimal psychological risks and personal time inconvenience. Participants will be 
compensated $175.00 per year for participating in a combination of survey, interview, and 
observation methods. Annual payments will be made at the end of spring semester upon 
continuous participation for that school year. 

All of the information that we gather on this project will remain strictly anonymous and will not 
identify you in any way. The interview sessions will be audio taped, and the ethnographic 
sessions will be audio taped, photographed, and videotaped so that we can review them later. 
However, all identifying information will be removed and replaced by code numbers. 
Participant codes will be kept in a logbook, which will be stored in a locked file cabinet separate 
from the actual data information. We will keep all materials in a secure place in our university 
office. 

The participants should be 18 years of age or older and in good health. If you are younger than 18, 
please contact the investigator immediately. 

The Principal Investigator and the Research Associate, may be contacted at [phone #] in the 
event that you have any questions regarding your participation in this project. If you have 
questions about your rights as a study participant, or are dissatisfied at any time with any aspect 
of this study, you may contact - anonymously, if you wish - the Human Subjects Division 
[contact info] . 
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ACADEMIC PATHWAYS STUDY WINTER 2006 
EORMAL INTERVIEW PROTOCOL 
TECHNICAL PUBLIC INSTITUTION/URBAN PRIVATE UNIVERSITY 
SUBURBAN PRIVATE UNIVERSITY/LARGE PUBLIC UNIVERSITY 

SECTION I. 

Instructions 

Good morning (afternoon, evening). My name is . Thank you for coming. This interview 

involves two parts. The first part is an interview in which I will ask you about your experiences 
as a student at this university. The purpose is to get your perceptions of your experiences inside 
and outside of the classroom. There are no right or wrong, or desirable or undesirable answers. 
I would like you to feel comfortable with saying what you really think and how you really feel. 
The second part is a short pencil-and-paper task, and I will give you specific instructions for 
completing that task once we have finished with the interview. The entire process should take 
approximately an hour to complete. 

The results of this research will provide useful information to engineering educators in 
structuring effective educational programs to help them succeed in college. 

Tape Recorder Instructions 

If it is okay with you, I will be recording our conversation to get all the details but at the same 
time be able to carry on an attentive conversation with you. I assure you that all your comments 
will remain confidential. I will be compiling a report, which will contain all participants’ 
comments, but have no references to individuals. 

Preamble/Consent Form Instructions 

Before we get started, please take a few minutes to read this preamble (Hand Respondent (R) 
preamble.) (After R returns preamble, turn tape recorder on.) 


Verbally ID the Recording 

1) Date; 2) Interviewer Name; 3) Time; 4) Location (Room# and School); 5) Type of Interview 
(Formal); 6) Respondent ID 


QI. What is your major? 

Qla. When did you declare this major? 
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Q2. Are you a member of any engineering student organizations on campus? 


I Yes I No ► Skip to Q3 

I 

Q2a. What are those organizations? 


Q2b. What are the most helpful aspects of that/those organization(s)? 


Q3. In your own words, would you please define engineering? 


Q4. Are there particular skills that you would say are important for an engineer to have? 



Q4a. (IF NOT ALREADY ANSWERED) What are those skills? 


Q4b. Of the skills that you mentioned, which ones do you possess? 


Q4c. Please tell me how you developed your skill(s)? 


Q5. Have you had any engineering internship experiences? 



Q5a. (IE NOT AEREADY ANSWERED) Please describe those experiences. 
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Q6. Have you had any experiences inside or outside of your classes that have enabled you to 
develop general engineering knowledge? 



Q6a. (IF NOT ALREADY ANSWERED) Please describe those experiences. 


Q7. Have you had any experiences inside or outside of your classes that have prevented you 
from developing general engineering knowledge? 


Yes 


No 


-►Skip to Q8 


Q7a. (IE NOT AEREADY ANSWERED) Please describe those experiences. 


Q8. Have you had any experiences inside or outside of your classes that have enabled you to 
solve problems? 



Q8a. (IP NOT AEREADY ANSWERED) Please describe those experiences. 


APS Research Processes and Procedures 
May 2009 


3C-15 









Q9. Have you had any experiences inside or outside of your classes that have prevented you 
from solving problems? 


Yes 


No 


■►Skip to QIO 


Q9a. (IF NOT ALREADY ANSWERED) Please describe those experiences. 


QIO. Where do you see evidence of your engineering abilities? 



Q1 1. On a scale from 0-10, (where 0 = not confident at all and 10 = extremely confident), 
how confident are you in your math ability? (ONEY ACCEPT WHOEE NUMBERS.) 

Q1 la. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK. ) 


Q12. On a scale from 0 - 10, (where 0 = not confident at all and 10 = extremely confident), 
how confident are you in your science ability? (ONEY ACCEPT WHOEE NUMBERS.) 

Q12a. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK.) 


Q13. On a scale from 0-10, (where 0 = not confident at all and 10 = extremely confident), 
how confident are you in your design ability? (ONEY ACCEPT WHOEE NUMBERS.) 
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Q13a. Describe the experiences that led you to rate yourself in this way. 
(REMIND THEM OE SCORE IE THEY ASK.) 


QI4. Are there any aspects of engineering that you particularly like? 

►Skip to Q15 


Yes 


No 


Q14a. (IE NOT AEREADY ANSWERED) What are some of those? 


Q15. Are there any aspects of engineering that you particularly dislike? 

Skip to Q16 



Q15a. (IE NOT AEREADY ANSWERED) What are some of those? 


Q16. In general, how do you feel about engineers? 

Q16a. (IE NOT AEREADY ANSWERED) And why? 


QI7. As a student, how do you identify with practicing engineers? 



Q18. How do you believe members of other professions feel toward engineers? 
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Q18a. (IF NOT ALREADY ANSWERED) And why? 


Q19. How important is being an engineering student to how you feel about yourself? 
Q19a. (IE NOT AEREADY ANSWERED) And why? 

Q20. How eommitted are you to pursuing an engineering major? 

Q20a. (IE NOT AEREADY ANSWERED) And why? 

Q21. What does diversity mean to you? 


Q22. To what extent do you eonsider your sehool to be diverse? 

Q22a. (IE NOT AEREADY ANSWERED) And why? 


Q23. Does your gender affeet your views of beeoming an engineer? 



Q23a. (IE NOT AEREADY ANSWERED) How? 


Q24. Does your raeial identity affeet your views of beeoming an engineer? 



Q24a. (IE NOT AEREADY ANSWERED) How? 
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Q25. Are any of your family members or close acquaintances working engineers? 


Skip to Q26 


Q25a. (IF NOT ALREADY ANSWERED) Who? 


Q25b. Did their experiences influence your decision to become an engineer? 


Skip to Q26 


Q25c. (IE NOT AEREADY ANSWERED) How? 


Q26. Do you have a mentor? 



Skip to Q27 


Q26a. (IE NOT AEREADY ANSWERED) How did you get affiliated with your 
mentor? 


Q26b. Has your mentor influenced your decision to continue majoring in engineering? 


Skip to Q27 



Q26c. (IE NOT AEREADY ANSWERED) How? 


Q27. When is your expected graduation date? 


Q27a. What do you see yourself doing after graduation? (Have R be specific) 
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Q28. At this institution, are there any aspects of majoring in engineering that are particularly 
difficult in achieving your academic goals? 



■■Skip to Q29 


Q28a. (IF NOT ALREADY ANSWERED) Please tell me about those difficulties. 


Q28b. How do you deal with those difficulties? 


Q29. At this institution, are there any aspects of majoring in engineering that are particularly 
helpful in achieving your academic goals? 


Yes 


T 


No 


^Skip 


to SECTION II 


Q29a. (IE NOT AEREADY ANSWERED) Please tell me about those helpful 
aspects. 
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Section 3: Formal Interview Debriefing 


(READ ALL OL THE LOLLOWING ALOUD TO R.) 

Thank you very much for coming this morning (afternoon, evening). Your time is very much 
appreciated and your comments have been very helpful. 

The purpose of this interview is to better understand students’ perceptions of their experiences 
inside and outside of the classroom. We are interested in your opinions and your reactions. In 
no way is this interview designed to individually evaluate a person’s abilities. The task is not 
diagnostic, nor can it provide a measure of the “quality” of your performance. Your only 
requirement was to do the best job that you could. 

Your identity will be kept anonymous during all phases of this study including any experimental 
writings, published or not. Procedures for maintaining confidentiality are as follows: (1) individual 
participants’ results will be pooled with group results; and (2) participants should not place any 
identifying information on data collection instruments. (Such identifiers include name, social 
security number, student identification number, specific birth data, telephone number, address, etc.) 

DQl. Is there any other information regarding your experience as an engineering major or at 
this institution that you think would be useful for me to know? 



Again, thank you for participating. (TURN TAPE-RECORDER OLE.) 
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Section 4: Formal Interview Interviewer Reflection 


Instructions: 

After the respondent leaves the room, please take a couple of minutes to indicate your reactions 
and observations about the interview. An electronic copy of this form has been provided. Feel 
free to use this hard copy for your own notes, but please submit the electronic copy for official 
use. 


Your name 
(the interviewer): 


Your race: 


Your gender: 


Your age: 



Respondent ID No.: 


Date of Interview: 


Please describe the 
respondent’s attitude 
toward you and the 
interview: 


Please describe any 
unusual 

circumstances and/ 
or events that had 
any bearing on the 
interview such as 
interruptions, 
language difficulty, 
etc.: 


Please describe 
anything else that 
happened during the 
interview that has 
any bearings on the 
study’s objectives: 


Additional 

comments: 



3C-22 


APS Research Processes and Procedures 

May 2009 






Probes for Formal Interview 


Interviewer’s Probe Abbreviation 


Repeat Question RQ 

Anything else? AE or Else? 

Any others? AO? 

How do you mean? How mean? 

Could you tell me more about your 

thinking on that? Tell more 

Would you tell me what you have 

in mind? What in mind? 

What do you mean? What mean? 

Why do you feel that way? Why? 

Which would he closer to the way 

you feel? Which closer? 
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Submission Instructions for Formal Interviews 


Formal Interview Protocol 

Be sure to write the Respondent’s ID Number on the first page of your Formal Interview 
Protocol sheet. (The paper document is the protocol sheet that the interviewer used during the 
interview and wrote notes on.) Keep the original protocol at your institution in a secure location. 

Digital Audio Files 

At the end of the interview, upload the audio file in the Interview Data folder of the CAEE 
Academic Pathways Database. Please read the section on “CAEE APS Database File 
Naming Protocol” for more information. Save a copy of the audio file on a compact disc to be 
kept at your institution. 

Please read the section on “The Transcription Process: For the Researcher” for more information. 
Interviewer Reflection 

Each interviewer will create an Interviewer Reflection document for each respondent. Here is an 
example of how this document would be named according to the File Naming Protocol: 

RespondentID-INTS-060329-3-Nl_l-KE.doc 


RespondentID 

Respondent ID 

INTS 

Eormal Interview 

Date 

Date in “YYMMDD” format 

2 

EventID indicating that this is data from the second formal interview 

Nl_l 

ItemID consisting of: Notes (Data Type), 1 (Item Number), 1 (Revision 
Number) 

KE 

ResearcherlD initials 

.doc 

Word file extension 


At the completion of all of the Formal Interviews, each school will have 24 Interviewer 
Reflection documents, one for each respondent. These 24 files should be zipped into one file 
using a program such as Aladdin’s Stuffit or WinZip. The one zipped file of each schools 
collection of Interviewer Reflections should be given a name similar to the following: 

SCH01-INTS-3-Nl_l.zip 


SCHOl 

School and Cohort 1 

INTS 

Eormal Interview 

3 

EventID indicating that this is data from the third formal interview 

N1_I 

ItemID consisting of: Notes (Data Type), 1 (Item Number), 1 (Revision 
Number) 

.zip 

Zipped file extension 


This one zipped file should be uploaded into the CAEE APS database. 
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Academic Pathways Study 
Ethnographic Interview Protocol 
Interviewer’s Guide 
January 23, 2006 


This document is intended as a guide to be used by interviewers in conducting the APS 
Ethnographic Interviews. It is not a “script.” Because ethnographic interviews are intended to 
be semi- structured and open-ended, the exact wording of the questions will change depending on 
the context. Furthermore, the questions below are not meant to “stand alone,” but rather to elicit 
responses that will be followed up on by the interviewer in order to explore the participant’s 
perspective further. 

The questions are listed in the order that we would like topics to be introduced. However, you 
might find that a participant brings up a topic on her/his own before the scheduled point in the 
interview. For example, a participant might mention issues of gender or ethnicity before you 
have reached these topics in Question 17. In this case, it is useful and perfectly appropriate to 
explore the topic when it is first introduced. (The later “slot” for that topic can then be used to 
address the area further, if in your judgment there is more insight into the participant’s 
perspective to be gained from this.) 

There is no scripted introduction to be used in the interviews with the 8 ethnographic participants 
- we want you to adopt the style, phrasing, and tone that are most comfortable for you. In 
general, we would suggest that you first introduce, or re-introduce, yourself. Remind the 
participant that this is the yearly interview that he or she has agreed to participate in, and that the 
interview will last approximately 2 hours. Tell the participant that you will be talking with 
her/him about a number of topics that are of interest for the Academic Pathways Study. These 
topics are about the student and his/her experiences in and out of school. Stress that you’re 
interested in the student’s ideas, opinions, and perspectives, and let her/him know that you 
appreciate whatever he/she is able to tell you about the topics you’ll be exploring. 

We have tried to indicate the kinds of follow up questions you might make in the context of the 
questions below. However, following up on evaluative statements is generally important, and 
warrants mentioning here. Whenever possible, follow up on subjects’ evaluative statements: 
e.g., S: “That was a hard class” . . .Int: “What was hard about it?” or “How was it hard?”; S: “I 
like math”... Int: “What do you l ik e about math? Can you give me a specific example?” 

Before interviewing a subject, remember to re-read and familiarize yourself with relevant data 
collected on that subject, including fieldnotes, the Year 1 ethnographic interview, and any 
informal interviews that may have been conducted. Also review the current state of the analysis 
of that subject’s data. This will allow you, on a case-by-case basis, to tailor questions to a 
particular subject. This will greatly increase the comparative power of the interviews for the 
longitudinal aspect of the study. 
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Please feel free to use your judgment to explore potential points of interest that might arise, 
whether or not they are included in the questions below. Sometimes the most important issues 
have not been anticipated by the researchers at the outset. 
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Academic Pathways Study Ethnographic Interview 
Interviewer’s Guide 
January 23, 2006 


1. Tell me how things are going for you. 

a. Explore further on specifics of response, if student doesn’t provide details: e.g., if 
student says, “It’s been a lot harder than I thought,” ask, “In what ways?” 

Note: Follow the subject’s lead on how to handle this question - in some cases, it might be best 
used as an icebreaker, just to begin the conversation and move on to other questions. However, 
some subjects ( most, based on our informal interviews at LPub this year) will have quite a lot to 
say, on a range of topics. In this case, don’t be too quick to move off of this question - explore 
all of the issues the subject seems willing to get into. 

b. Before moving on to other questions, give subject a chance to say more, e.g., “What 
else is happening with you these days?” 


2. We talked about this some last year, but I’d l ik e to ask about it again: Can you tell me how 
you became interested in engineering? 


a. Explore further, if needed, with: What were some of the experiences that were 

important in getting you interested in engineering? 


3. Thinking about yourself before you came to [School Name] - are there things that you would 
say prepared you to succeed in engineering? 


4. What are you majoring in (or planning to major in)l 


a. How did you choose (xxx)? 


b. What other majors did you consider? 


c. What other kinds of engineering did you consider? 


d. Are there certain fields of engineering that you just can’t see yourself going into? 

Why? 
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5. Can you take me through a typical day for you here at [School Name]? 


6. I’d like to ask about your classes and other academic experiences you’ve had since you’ve 
been here. 


a. Have you had academic experiences here so far that you would describe as particularly 

good? Can you tell me about one of those? If necessary, make sure to follow up with a question 
like: “So what was it that made that a good experience?” 


b. Have you had academic experiences that you would describe as particularly bad? Can 

you tell me about one of those? Again, if necessary, make sure to follow up with a question like: 
“So what was it that made that a bad experience?” 


7. How would you describe yourself as a student? 


a. Make sure to explore further on issues of change and development brought up by the 

subject, e.g., “I’m a lot different now than I was last year. ” 


8. Let me ask you to think about the other engineering students you’ve come across here. Would 
you say that in general they are more different from you or more similar? 


a. Explore further: “How are they similar?” (or “How are they different?”) 


9. Have you had much contact with non-engineering students? Would you say that there are 
things that distinguish engineering students from students in other majors? 


a. Explore further as needed to get details. 
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10. What would you say has been the most difficult thing here for you so far? [Students might 
mention non-academic dijficulties. Make sure to explore further to get at academic difficulties.] 


a. How did you handle (or how are you handling) that? 

The following sub-questions can be asked if time permits. 

b. What else have you found difficult? 

i. How have you handled that? 

c. Do you have strategies for handling difficult situations? 

i. Explore further for specific examples. 

11. What’s been easy for you here so far? 

a. Explore further if subject doesn ’t elaborate. 

12. Has anything surprised you about your classes? 

13. Let me ask you about your strategies for deciding what classes to take. 

a. What kinds of things do you think about when you’re deciding what classes to take? 

b. Do you talk to other people when you’re making your decisions about classes? 
i. Explore further to get at who these other people are as well as the role they play. 

Note: Be prepared to explore further on issues such as the following: what is an easy (or good, 
or interesting, etc.) course, what is a hard (or bad, or boring, etc.) course; what’s an easy (or 
good, etc.) professor, what’s a hard (or bad, etc.) professor; and how do they know about 
particular professors, courses, etc. 
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14. Are there any groups that you’ve become part of since you came here to [Name of 
Institution]? 


a. Explore further for each group (or, if there are a lot, for what the subject would say 

are the “most important” ones) 

i. Can you tell me about [group]? 

Note: This may need some prompting to establish the range of places we are interested in, both 
on and off campus, both social and para-professional. Explore further to see if they have 
answered with respect to both on and off campus groups. 


15. Think about your professors here at [Name of Institution] . What would you say they think it 
means to be a good engineer? 

a. How does that fit with your own image of a good engineer? 


16. Knowing what you know now, as you look back on the time that you’ve spent here, is there 
anything that you would do differently? 


17. One of the things that our research team is interested in is diversity in engineering and 
engineering education, in terms of race, ethnicity, and gender. I’d like to ask you some questions 
related to this. 


a. (This is a question that you’ve already answered on the survey, but your survey 

responses aren’t available to me yet.) Can you tell me how you identify yourself racially or 
ethnically? 


b. Are there supports or barriers, advantages or disadvantages, for you as a [ethnic 

identification] engineering student? 

i. How about for people of other racial or ethnic groups? 
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c. Do you think that there are differences between the experiences of male and female 

engineering students? 


d. How has it been for you here, as a [male/female] engineering student? 


18. 1 want to ask you to think about the rest of the time that you’ll be spending at [School 
Name]. 


a. What are you looking forward to during the rest of your time here? 


b. What are you concerned about? 


19. Okay, let’s imagine it’s a few years from now, and you’ve graduated with a degree in 
(student’s planned major). 


a. What’s next for you? 

i. Or, if not planning on becoming an engineer, explore why they’ve made this decision. 


b. What do you imagine yourself doing on a day-to-day basis? 

i. Or, if not planning on becoming an engineer: What do you imagine 
engineers do on a day-to-day basis? 

c. What would you say it takes to be a good (insert student’s career choice)? 

d. How are you at (insert characteristics student mentions)? 


e. Are there things about yourself that you think you need to work on to become a 

successful (xxx)? 
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20. Here’s a more personal question. I’ve been asking a lot of questions so far about your 
academic experiences. What else do you do in your life besides being an engineering student? 


(Question 23 is partially redundant with question 17; ask if time permits.) 

21. Here’s a scenario I want to ask you about: There’s a high school student who’s interested in 
pretty much what you were interested in when you were in high school. This student comes to 
you for advice. Knowing what you now know, what advice would you give her/him? 


22. If you could whisper in the ear of the people who set up the pre-engineering (or engineering) 
program here at [Name of Institution], what advice would you give them about improving 
things? 


23. Is there anything that I haven’t asked you about that you think I should? Anything else that’s 
relevant in an interview like this, do you think? 


24. Do you have any questions you’d like to ask me? 
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Submission Instructions for Ethnographic Interviews 


Ethnographic Interview Protocol 

Be sure to write the Respondent’s ID Number on the first page of your Ethnographic Interview 
Protocol sheet. (The paper document is the protocol sheet that the interviewer used during the 
interview and wrote notes on.) Keep the original protocol at your institution in a secure location. 


Digital Audio Eiles 

At the end of the interview, upload the audio file in the Interview Data folder of the CAEE 
Academic Pathways Database. Please read the section on “CAEE APS Database File 
Naming Protocol” for more information. Save a copy of the audio file on a compact disc to be 
kept at your institution. 
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Section 2: Performance Task Background 
Performance Task Protocol and Respondent Question Sheet 


Respondent ID: 

Performance Task 

Transition for Ethnographic Interviews: 

The last part of today’s session has a different format from the interview we just completed. I 
will be reading instructions and questions from a script. 

Instructions: 

At this time, I'd like to ask you to work on a short activity. While I hope that it is a fun activity for 
you, I would also like you to give it your best effort. You have up to ten minutes to work on it. 
Please let me know if you are done before that. Do you have any questions? 

OK, here is the activity — I’m going to read it with you, out loud. You might remember this 
activity if you were asked to do it two years ago as part of this study. (CHECK START TIME 
ON AUDIO RECORDER AND INDICATE IT IN THE BOX BEEOW. THEN HAND R THE 
PEREORMANCE TASK EORM.). 


Start time: 


HH MM SS 


PTQI : (NEXT READ THE EOEEOWING AEOUD TO R) Over the summer the Midwest 

experienced massive flooding of the Mississippi River. What factors would you take 
into account in designing a retaining wall system for the Mississippi? 

Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): Okay, it’s been 10 
minutes now, please stop. 


End time: 


HH MM SS 


PTQ2: What questions came to your mind as you were brainstorming your list? 


(THE QUESTIONS MUST BE EUEEY EORMUEATED. IE R OEEERS A ERAGMENT AS A 
QUESTION, REMIND R TO SPEAK AS IE HE/SHE IS PEAYING JEOPARDY, AND ASK R 
TO CEARIEY HOW HE/SHE USED THE ERAGMENT IN A QUESTION. IE R IS NOT 
EAMIEIAR WITH JEOPARDY, TEEE R THAT IT IS OKAY, AND THAT AEE HE/SHE 
NEEDS TO DO IS TO RESPOND IN QUESTIONS ONEY. IT IS OKAY EOR R TO 
BROWSE THROUGH THE EIST OE SOEUTIONS HE/SHE HAS WRITTEN, BUT THERE 
IS NO NEED TO SUGGEST THIS IN YOUR INSTRUCTIONS. IE R OEEERS 2 
QUESTIONS OR EESS, PROMPT HIM/HER AGAIN. R SHOUED BE ABEE TO PROVIDE 
5-10 QUESTIONS IN 2-3 MINUTES.) 
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(IF YOU HAVE A YEAR 1 MIDWEST EEOODS WRITTEN RESPONSE EOR R, GIVE IT 
TO THEM AND PROCEED TO PTQ3. IE NOT, SKIP AHEAD TO PTQ5, WHICH IS 
MARKED WITH A T.) 


Here’s a copy of your list of factors from back in 2004, when you first did the Mississippi 
flooding activity. 

PTQ3: Take a look at both the response you just wrote today and your response from two 

years ago. What similarities and differences do you notice between the two responses? 


PTQ4: You ’ve told me a little about how your responses are similar or different. How about 

how you came up with them ? Consider how you thought about the activity and how 
you came up with the factors you wrote down, both today and two years ago. What 
similarities and differences do you notice? 

(IT’S EINE IE R AEREADY BEGAN COMPARING THOUGHT PROCESSES (VS. 
COMPARING RESPONSES) IN ANSWERING THE PREVIOUS QUESTION (PTQ3). ASK 
THEM TO CONTINUE, E.G., “Do you notice any other similarities or differences in the way 
you came up with your response?”) 


T 

PTQ5: Have you had any past experiences that helped you do the written activity? 

(IE SO, ASK R TO DESCRIBE THE EXPERIENCES.) 


PTQ6: Have you had any educational experiences that helped you do this activity? 

(IE SO, ASK R TO DESCRIBE THE EXPERIENCES. R MIGHT HAVE AEREADY 
DISCUSSED EDUCATIONAE EXPERIENCES IN RESPONDING TO PTQ5. IP SO, ASK 
THEM TO CONTINUE, E.G., “Aro there any other educational experiences that helped you do 
the activity?”, OR ASK EOR ADDITION AE DETAIES ABOUT THE EDUCATIONAE 
EXPERIENCES, IP TIME PERMITS.) 
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(IF YOU DID HAVE A YEAR 1 MIDWEST EEOODS WRITTEN RESPONSE EOR R, SKIP 
AHEAD TO THE CLOSING TRANSITION BELOW, MARKED WITH A ■.) 


Transition for students who did not do the activity in Year 1: 

Now, I’d like to ask you some questions about a recent natural disaster in the U.S. 

PTQ7: How familiar are you with Hurricane Katrina and the flooding in New Orleans? 

Could you tell me what you know about these events? 


PTQ8: Did what you know about these events affect how you approached the Mississippi 

flooding activity today? 

(IE SO, ASK R TO DESCRIBE HOW THEIR KNOWLEDGE AEEECTED THEIR 
APPROACH TO THE ACTIVITY.) 


Closing transition eor all students: 

(COLLECT BOTH 2004 AND TODAY’S RESPONSE PAGES.) Great! Thank you. We hope 
that you ’ve enjoyed this activity and we want to make sure that you know that there are many 
right answers. We’ve used it to collect information from engineering students across the nation 
to understand the types of things students think about. 
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Over the summer the Midwest experienced massive flooding of the Mississippi 
River. What factors would you take into account in designing a retaining wall 
system for the Mississippi? 
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Submission Instructions for the Performance Task 

Timing : Jot down the approximate start and end times while administering the task. After 
the respondent has left, double check the start and end times by reviewing the audio 
recording. Record the start and end times in minutes and seconds (e.g., start 43:21, end 
49:16) on the Performance Task Sheet (the sheet where respondent has listed factors). 

Labeling : At the end of the formal interview, write the Respondent ID and the start and 
stop times on the Performance Task Sheet. 

What to do with the Performance Task Response Sheets : Make a photocopy, keep the 
photocopy in a secure location, and then send the original (via FedEx/UPS/or some other 
service that will track the package) to: [researcher contact information] 


You may send the Performance Task sheets to the researcher in one batch. 
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Academic Pathways Study 
Exit Interview Protocol 

Technical Public Institutlon/Urban Private University/ 
Suburban Private University 
2005-2006 

1 . Can you tell me about your reasons for majoring in engineering when you started at 
(name of school)? 

Note: For this question in particular, it will be worth knowing how a given subject 
has answered this question (or similar questions) previously. We wouldn ’t suggest 
being confrontational (i.e., “But wait, that’s not what you said last year... ’’), but 
previous responses can usefully inform follow-ups for this question. 

This can be a very useful question analytically, in that it can be compared to what 
students said in their Year 1 interviews. We expect that students might answer this 
question retrospectively, from the perspective of someone having left. 

Please probe the answer to gauge whether the student had a prior connection to 
engineering through family or friends or some other type of exposure. Did the 
student’s understanding of what engineering would be like change? Question #4, 
below, gets at this as well. 


2. Think about your expectations when you started last year. Did things go as you 
expected? What did? What didn’t? 

3. What’s your current major? How did you choose XXX? How is it going? 

4. a. When did you first start to think about switching out of engineering? (or, if more 

appropriate, into their current major - some students might frame leaving 
engineering as a positive switch into something that was more appealing to them.) 


Follow up with, e.g., “What was going on at the time to make you start considering 
this?, ” and explore each of the things they discuss. 

4.b. So from that point, can you talk me through the process until now? 

Follow up with, e.g., “What was switching like for you? Were there difficult 
moments? Were there moments when you doubted the direction you were headed in? 
Were there moments when you were certain you were headed in the right direction? 
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Were there any particularly important events/moments/etc.?” Any specifics should 
be explored further for more detail. 


5. a. I’d like to ask about some of your specific experiences. Let’s start with your classes. 
Did you have any classes that you would say were particularly good ones? (Follow up on 
why they were good, looking for details.) Did you have any classes that you would say 
were particularly bad ones? (Follow up on why they were good, looking for details.) 


5.b. How about out of class? Any particularly good/bad experiences? (Follow up as 
above) 

Note: If student volunteers non-academic out of class experiences, let them talk about 
those. We’d suggest then directing them back to out of class academic experiences 
(e g., tutoring, office hours, etc.) The mentor questions might come up here as 
well. 


6. How would you say you’re feeling at this point about leaving engineering? 

7. What’s different for you now that you’re (no longer an engineering major/ an xxx 
major)? 

8. What’s ahead for you? What are you planning for when you leave (name of school)? 

9. If you could talk to the people who run the engineering program, what would you tell 
them? 

10. Do you have anything to add to what you’ve said so far? 
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Academic Pathways Study 
Exit Interview 

Large Public University 
2005-2006 

1 . Can you tell me about your reasons for intending to major in engineering when you 
started at LPub? 

la. Were you aware of the engineering admission process? 

Note: For this question in particular, it will be worth knowing how a given subject has 
answered this question (or similar questions) previously. We wouldn’t suggest being 
confrontational (i.e., “But wait, that’s not what you said last year...”), but previous 
responses can usefully inform follow-ups for this question. 


This can be a very useful question analytically, in that it can be compared to what 
students said in their Year 1 interviews. We expect that students might answer this 
question retrospectively, from the perspective of someone having left. 

Please probe the answer to gauge whether the student had a prior connection to 
engineering through family or friends or some other type of exposure. Did the 
student’s understanding of what engineering would be like change? Question #4, 
below, gets at this as well. 


2. Think about your expectations when you started last year. Did things go as you 
expected? What did? What didn’t? 

3. What’s your current major? How did you choose XXX? How is it going? 

4. a. When did you first start to think about switching out of engineering? {or, if more 

appropriate, into their current major - some students might frame leaving 
engineering as a positive switch into something that was more appealing to them.) 

Follow up with, e.g., “What was going on at the time to make you start considering 
this?, ” and explore each of the things they discuss. 

4.b. So from that point, can you talk me through the process until now? 

Follow up with, e.g., “What was switching like for you? Were there difficult 
moments? Were there moments when you doubted the direction you were headed in? 
Were there moments when you were certain you were headed in the right direction? 
Were there any particularly important events/moments/etc.?” Any specifics should 
be explored further for more detail. 
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5. a. I’d l ik e to ask about some of your specific experiences. Let’s start with your 
classes. Did you have any classes that you would say were particularly good ones? 
(Follow up on why they were good, looking for details.) Did you have any classes that 
you would say were particularly bad ones? (Follow up on why they were good, looking 
for details.) 


5. b. How about out of class? Any particularly good/bad experiences? (Follow up as 
above) 

Note: If student volunteers non-academic out of class experiences, let them talk about 
those. We’d suggest then directing them back to out of class academic experiences 
(e g., tutoring, office hours, etc.) The mentor questions might come up here as 

well. 

6. How would you say you’re feeling at this point about leaving engineering? 

7. What’s different for you now that you’re (no longer an engineering major/ an xxx 
major)? 

8. What’s ahead for you? What are you planning for when you leave (name of school)? 

9. If you could talk to the people who run the engineering program, what would you tell 
them? 

10. Do you have anything to add to what you’ve said so far? 
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CAEE APS Database File Naming Protocol 


This information is taken directly from the Administrativia section of the APS workspace 
on the CAEE Database. For questions or more information, please contact the database 
administrator. 


Data File Naming 


Names of data file within the APS database as proposed here has 6 standard components (plus 
one optional component,) organized in the following order: 


1. StudentID: coded per CAEEI D 

2. MethodType: instrument or method 

3. EventID: event_sequence or event_date, or combination 

4. ItemID: item_type, item_number and revision_number 

5. Researcherl D: name initials 

6. (optional) Pseudonym: reference subject's "name" 

7. FilenameExtension: document_type 


To improve readability and facilitate accurate computer-based parsing, filename components 
are separated by hyphens. By design, the resultant filename will uniquely identify the context 
of each data file in the APS database. This filenaming convention would result in filenames 
that look like 

StudentI D - MethodType - EventI D - ltemI D - Researcherl D - Pseudonvm . FExt 


As examples, we may have: 

TPub01F00003-INTS-3-Al_l-GT.dss 

UPri01F00025-INTS-3-Tl_3-KE.rtf 

SPri01F00008-ETH-060306-Nl_l-TLB.rtf 

LPub01M00034-INSP-3-Sl_l-KO.pdf 


This document is a work in progress and may be revised to reflect new needs and functions of 
the APS database. It is important that this proposed filenaming protocol, not only meet the 
immediate needs related to the current study of cohort- 1, but can also be carried forward to 
integrate data for future APS cohort studies. 

StudentID 

In the APS database, we must avoid identifying our study's student participants by real names 
or other recognizable real-world information, in association with collected data. To abstract a 
participant's identity, we have developed a coding scheme. This CAEE Student ID code 
uniquely identifies each student participant and can be broken down into 4 parts. It looks 
something like: "TPub01F00003". The first part, "TPub", is the school's official acronym (i.e. 
Technical Public Institution). The second part, "01", is the cohort id. The third part, "F", refers 
to the gender, female. The fourth an final part, "00003", is a sequentially generated number 
identifier of the student at the indicated school. 

For cohort-1, we have TPub, UPri, SPri, and LPub as the 4 possible school acronyms. When 
expanding the study in cohort-3, and cohort-4, most U.S. schools have unique acronyms. If 
we should encounter two or more schools with identical acronyms, we can append a lower 
case letter (a,b,c. ..) in sequence to differentiate these schools. Internal to the APS database, 
there is a table that connects this acronym (known as UnivI D) to the full name of the school 
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and its related descriptive data (e.g., university's full name, semester/quarter system, etc.). 

With this StudentID code displayed in the filename, a researcher can quickly tell at a glance 
that the data file is associated with a specific student who attends a given school, is of a given 
gender, and participates as a member of a specific cohort. 

If student data were aggregated into container documents by school, the containers' filenames 
would only include the school acronym and the cohort id (e.g. "TPubOl"). No gender and 
student identifying sequence number would be included. 

MethodType 


The MethodType component in the filename identifies the data instrument or method being 
used to collect the data contained in the file. This portion of the filename is typically 3-4 
letters long. For our research as planned, the following MethodTypes would be used: 


SURV 

survey data 

INTS 

structured interview data, 

INTE 

ethnographic interview data 

INTX 

exit interview data 

INSP 

problem scoping exercise data within structured interview 
session 

PERF 

performance task data 

ETH? 

ethnography data 


EventID 

The EventID is used to identify the particular instance of the data collection event. The 
EventID, taken together with MethodType, allow us to refer to a specific data set in a 
sequence, such as Survey 2, or Structured Interview 1. We will use numeric digits such as {1, 
2, 3} to specify the EventID. 

Ethnographers will typically contribute new files on a regular basis throughout the year. It may 
be more appropriate for such research methods to use date in lieu of a sequence number to 
represent the data gathering event. When applied, the date would be formatted as "YYMMDD", 
so as to facilitate chronological sorting. This date information should not substitute for the 
inclusion of more detailed date information inside the file document itself. 

ItemID 

The ItemID is used to identify one or more data items collected together within the context of 
a single data collection event. It is composed of three parts, in order: 

1. DataType, 

2. ItemNumber, and 

3. RevisionNumber. 

For example, in the course of structured interview #1, we may produce one audio recording 
file, one interview notes document, and 2 PDF scan files. In this scenario, we would have 4 
files with 

MethodType = I NTS 
EventI D=1 

and the following distinguishing ItemIDs 
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ltemlD=Al_l (Audio File) 
ltemlD=Nl_l (Notes File) 
ltemlD=Sl_l (Scan File #1) 
ltennlD=S2_l (Scan File #2) 

If after review, the notes file with ItemI D=Nl_lwas revised and resubmitted to the database, 
the revised file would take on ItemI D=N1_2. 

At the current time, the following DataType codes are proposed: 


A 

audio recordings 

V 

video recordings 

N 

notes (field notes, interview notes) 

T 

text transcriptions 

R transcript revision requests 

X 

delimited columnar data 

P 

digital photos 

S 

scanned paper documents 


ResearcherlD 

The ResearcherlD identifies the researcher who is primarily responsible for collecting the data 
in the file. The researcher's initials (in all capital letters) will be used. If we should encounter a 
situation in which a new researcher has initials identical to an existing researcher, we would 
append a number to the new researcher's initials for the ResearcherlD. For example, if we 
have a researcher Gwendelyn Talbot and we add a new researcher Greg Taylor, Gwendolyn 
Talbot would have ResearcherlD "GT" and Greg Taylor would be assigned "GTl". 

Pseudonym 

Ethnographers will typically contribute new files on a regular basis throughout the year. In 
such cases, the CAEE student ID may be hard to write and refer to in discussion. The student 
pseudonym, appended to the root part of the document name and also recorded in the APS 
database, would be the identifier that ethnography researchers would use to refer to the 
student subject. 

Student pseudonyms would be created by the research teams at each university, and may be 
something like "Legol". Each pseudonym is unique within the subject group at each school; it 
is not permitted to have another "Legol" within that school's subject group. However, it is 
entirely acceptable to have "Legol" at another school. This way, researchers have full 
autonomy and flexibility to choose pseudonyms without fear of conflicts with those chosen by 
researchers at other universities. 

Pseudonym, this filename component, is optional and is not likely used outside of ethnography 
documents. 

FilenameExtension 

We use familiar filename extensions to identify their respective applications and data types. 
The following example filename extensions would be used with associated applications: 


.rtf 

Microsoft Word, Mac TextEdit 

b. 

o 

n 

Microsoft Word 

■ dot 

Microsoft Word (Template) 
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■ Xls 

Microsoft Excel (spreadsheet) 

■ CSV 

Microsoft Excel (comma separated values) 

■ dss 

Olympus Audio Recorder/ Player 

.pdf Adobe Acrobat Reader, Mac Preview 

.txt 

Microsoft Notepad, Mac TextEdit 

■ zip 

■jpg 

WinZip, Windows Explorer (XP), Stuffit Expander 

Internet Explorer, Netscape, Mozilla, Safari 

.tif 

(tagged image format) 
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The Transcription Process: For the Researcher 


For questions or more information, please contact the APS Librarian. 

The two main steps for the transcription process are: 

I. File Uploading 

a. File Naming Protocol 

II. Reviewing returned transcripts from transcription company 


I. File Uploading 

Once an interview has been recorded in the .dss format, the following steps need to he taken: 

1 . Login to the APS database 

2. Click on the “Interview Data” folder that is on the left side-bar menu (see below) 


Project: APS 


Home 

Administrivia 
Schedule of Activity 
Study Schools 
Project Materials 
Library 



□ Ethiwgraphy Data 


Transcription Service 
Discussions 
Suggestion Bdk 
T eam Members 
Recent Changes 

Q My W-Grk£jia£-& 


3. Inside this folder, there will be the following areas: 

a. “In” Area 

b. Return Area 

c. Final Area 

d. Transcription Area 

4. Click on the Inbox for all interview audio files 
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5. This is the area where all NEW interview audio files (in the .dss format) need to he 
placed. Every Eriday at 2pm, this area will he checked for new files and all new files 
will he organized into a hatch to he sent to the transcriber. Researchers should upload 
all new interview files at the end of each day to this area. PLEASE REMEMBER to 
use the appropriate file naming protocol as specified. 

6. Once the file has been uploaded, please write a “Description” which includes the 
following information (please put each piece of information on a separate line): 

a. CAEE ID 

b. Researcher 

c. Misc info and notes (if needed) 

II. Reviewing Returned Transcripts from Transcription Service 

1 . Once the interview audio files have been sent to the transcriber, transcriber will 
return the files as interview transcripts in Rich Text Eormat (.rtf files). These files 
will be placed in the Return Area, sorted by school and placed in the respective 
school’s folder. 

2. A mass email notification will be sent to all of the researchers notifying them that 
new transcripts have been returned from transcriber and are ready to be reviewed. 

3. Researchers should review only their transcript files. The transcript files will be in .rtf 
format, and the original file names will be kept. Researchers should be able to 
identify which files are theirs by seeing the appropriate initials in the file name, as 
specified by the file naming protocol. 

4. Review the transcript. 

1 . If no changes need to be made, then email the transcriber that the 
transcript file is ready to be finalized. Once finalized, the 
transcript will be placed in the Einal Area inside the Einal 
Transcripts folder. 

2. If the transcript needs to be completely re-transcribed, then make 
notes about the observed problems and re-submit the transcript 
(in .rtf format), the original audio file, and any notes taken about 
the problems or changes that need to be made. Then place these 
files (still using the file naming convention) into the Re- 
submission inbox, located inside the Inbox for all interview 
audio files. These re-submitted files will go through the same 
process with the transcriber as the original files. 

5. All of the original audio files will be located in the Audio Archive in the Einal Area. 

6. All versions of transcript revisions and notes that have been re-submitted to the 
transcriber will be located in the Transcript Revisions & Notes folder in the Einal 
Area. 
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APS Semi-structured Ethnographic Interview Guide Exampies 


Interviewer's Guide (2004) J-2 

Interviewer's Guide (2007) J-7 
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Academic Pathways Study Ethnographic Interview Protocol 
Interviewer’s Guide 
March 7, 2004 


This document is intended as a guide to be used by interviewers in conducting the APS 
Ethnographic Interviews. It is not a “script.” Because ethnographic interviews are intended to 
be semi- structured and open-ended, the exact wording of the questions will change depending on 
the context. Furthermore, the questions below are not meant to “stand alone,” but rather to elicit 
responses that will be followed up on by the interviewer in order to explore the participant’s 
perspective further. 

The questions are listed in the order that we would l ik e topics to be introduced. However, you 
might find that a participant brings up a topic on his or her own before the scheduled point in the 
interview. For example, a participant might mention issues of gender or ethnicity before you 
have reached these topics in question 18. In this case, it is useful and perfectly appropriate to 
explore the topic when it is first introduced. (The later “slot” for that topic can then be used to 
address the topic further, if in your judgment there is more insight into the participant’s 
perspective to be gained from this.) 

There is no scripted introduction to be used in the interviews with the 8 ethnographic participants 
- we want you to adopt the style, phrasing, and tone that is most comfortable for you. In general, 
we would suggest that you first introduce, or re-introduce, yourself. Remind the participant that 
this is the yearly interview that he or she has agreed to participate in, and that the interview will 
last approximately 2 hours. ^ Tell the participant that you will be talking with them about a 
number of topics that are of interest for the Academic Pathways Study. These topics are about 
themselves and their experiences in and out of school. Stress that you’re interested in their 
ideas, opinions, and perspectives, and let them know that you appreciate whatever they’re able to 
tell you about the topics you’ll be exploring with them. 

Please feel free to use your judgment to explore potential points of interest that might arise, 
whether or not they are included in the questions below. Sometimes the most important issues 
are those that aren’t anticipated by the researchers at the outset. 


The ethnographic interview will last less than 2 hours with the 8 participants who will be participating in both the 
formal and the ethnographic interviews. 
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Academic Pathways Study Ethnographic Interview 
Interviewer’s Guide 
March 7, 2004 


1. Let’s start by talking a bit about your time in high school. 


a. Where did you go to high school? NOTE: This is intended as a warm up 

question, just to get the interview underway, 
and can be followed up with a few other 
“small talk” questions, like “Where is that?” 
etc. 


b. How would you describe yourself as a student in high school? 


2. Can you tell me how you decided to come to [School Name]? 


3. Can you tell me how you became interested in engineering? 


a. Explore further, if needed, with: What were some of the experiences that were 

important in getting you interested in engineering? 


4. Thinking about yourself before you came to [School Name] - are there things that you would 
say prepared you to succeed in engineering? 


5. Tell me about your first year at [School Name] so far. How are things going? 

a. Explore further on specifics of response, if student doesn’t provide details: e.g., if 

student says, “It’s been a lot harder than I thought,” ask, “In what ways?” 


6. Can you take me through a typical day for you here at [School Name]? 
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7. I’d like to ask about your classes and other academic experiences you’ve had since you’ve 
been here. 


a. Have you had academic experiences here so far that you would describe as 
particularly good? Can you tell me about one of those? If necessary, make sure to 
follow up with a question like: “So what was it that made that a good 
experience?” 

b. Have you had academic experiences that you would describe as particularly bad? 
Can you tell me about one of those? Again, if necessary, make sure to follow up 
with a question like: “So what was it that made that a bad experience?” 


8. 1 asked you earlier to describe the student you were in high school. How would you describe 
yourself as a student now? 


a. If relevant, explore further to get student to account for the differences. 


9. Let me ask you to think about the other engineering students you’ve come across here. Would 
you say that in general they are more different from you or more similar? 


a. Explore further: “How are they similar?” (or “How are they different?”) 


10. Have you had much contact with non-engineering students? Would you say that there are 
things that distinguish engineering students from students in other majors? 


a. Explore further as needed to get details. 
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1 1 . What would you say has been the most difficult thing here for you so far? [Students might 
mention non-academic dijficulties. Make sure to explore further to get at academic difficulties.] 

a. How did you handle (or how are you handling) that? 

The following sub-questions can be asked if time permits. 

b. What else have you found difficult? 

i. How have you handled that? 

c. Do you have strategies for handling difficult situations? 

i. Explore further for specific examples. 

12. What’s been easy for you here so far? 

a. Explore further if subject doesn ’t elaborate. 

13. Has anything surprised you about your classes? 


14. Let me ask you about your strategies for deciding what classes to take. 


a. What kinds of things do you think about when you’re deciding what classes to 
take? 


b. Do you talk to other people when you’re making your decisions about classes? 

i. Explore further to get at who these other people are as well as the role they 
play. 

Note: Be prepared to explore further on issues such as the following: what is an easy (or 
good, or interesting, etc.)) course, what is a hard (or bad, or boring, etc.) course; what’s 
an easy (or good, etc.) professor, what’s a hard (or bad, etc.) professor; and how do 
they know about particular professors, courses, etc. 
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15. Are there any groups that you’ve become part of since you came here to [Name of 
Institution]? 


a. Explore further for each group (or, if there are a lot, for what the subject would 

say are the “most important” ones) 

i. Can you tell me about [group]. 

ii. What role has [group] played in your education? 

Note: This may need some prompting to establish the range of places we are interested 
in, both on and off campus, both social and para-professional. Explore further to see if 
they have answered with respect to both on and off campus groups. 


16. Think about your professors here at [Name of Institution] . What would you say they think it 
means to be a good engineer? 

a. How does that fit with your own image of a good engineer ? 


17. Knowing what you know now, as you look back on the time that you’ve been here so far, is 
there anything that you would do differently? 


18. One of the things that our research team is interested in is diversity in engineering and 
engineering education, in terms of race, ethnicity, and gender. I’d like to ask you some questions 
related to this. 


a. (This is a question that you’ve already answered on the survey, but your survey 
responses aren’t available to me yet.) Can you tell me how you identify yourself 
racially or ethnically? 

b. Are there supports or barriers, advantages or disadvantages, for you as a [ethnic 
identification] engineering student? 

i. How about for people of other racial or ethnic groups? 

c. Do you think that there are differences between the experiences of male and 
female engineering students? 
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d. How has it been for you here, as a [male/female] engineering student? 

19. What are you planning to major in? 

a. How did you ehoose (xxx)? 

b. Have you considered other majors? 

c. Have you considered other kinds of engineering? 

d. Are there certain fields of engineering that you just can’t see yourself going into? 
Why? 

20. Let me ask you to think about the next several years that you’ll be spending at [School 
Name]. 


a. What are you looking forward to in your time here? 


b. What are you concerned about? 


21. Okay, let’s imagine it’s a few years from now, and you’ve graduated with a degree in 
(student’s planned major). 


a. What’s next for you? 

i. Or, if not planning on becoming an engineer, explore why they’ve made 
this decision. 


b. What do you imagine yourself doing on a day-to-day basis? 

i. Or, if not planning on becoming an engineer: What do you imagine 

engineers do on a day-to-day basis? 


c. What would you say it takes to be a good (insert student’s career choice)? 
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d. How are you at (insert characteristics student mentions)? 


e. Are there things about yourself that you think you need to work on to become a 

successful (xxx)? 


22. Here’s a more personal question. I’ve been asking a lot of questions so far about your 
academic experiences. What else do you do in your life besides being an engineering student? 


(Question 23 is partially redundant with question 17; ask if time permits.) 

23. Here’s a scenario I want to ask you about: There’s a high school student who’s interested in 
pretty much what you were interested in when you were in high school. This student comes to 
you for advice. Knowing what you now know, what advice would you give them? 


24. If you could whisper in the ear of the people who set up the pre-engineering (or engineering) 
program here at [Name of Institution], what advice would you give them about improving 
things? 

25. Is there anything that I haven’t asked you about that you think I should? Anything else that’s 
relevant in an interview like this, do you think? 


26. Do you have any questions you’d like to ask me? 
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Academic Pathways Study Ethnographic Interview 
Interviewer’s Guide 
February 23, 2007 

This document is intended as a guide to be used by interviewers in conducting the APS 
Ethnographic Interviews. It is not a “script.” Because ethnographic interviews are intended to 
be semi- structured and open-ended, the exact wording of the questions will change depending on 
the context. Furthermore, the questions below are not meant to “stand alone,” but rather to elicit 
responses that will be followed up on by the interviewer in order to explore the participant’s 
perspective further. 

The questions are listed in the order that we would l ik e topics to be introduced. However, you 
might find that a participant brings up a topic on her/his own before the scheduled point in the 
interview. For example, a participant might mention issues of gender or ethnicity before you 
have reached these topics in Question 17. In this case, it is useful and perfectly appropriate to 
explore the topic when it is first introduced. (The later “slot” for that topic can then be used to 
address the area further, if in your judgment there is more insight into the participant’s 
perspective to be gained from this.) 

There is no scripted introduction to be used in the interviews with the 8 ethnographic participants 
- we want you to adopt the style, phrasing, and tone that are most comfortable for you. In 
general, we would suggest that you first introduce, or re-introduce, yourself. Remind the 
participant that this is the yearly interview that he or she has agreed to participate in, and that the 
interview will last approximately 2 hours. Tell the participant that you will be talking with 
her/him about a number of topics that are of interest for the Academic Pathways Study. These 
topics are about the student and his/her experiences in and out of school. Stress that you’re 
interested in the student’s ideas, opinions, and perspectives, and let her/him know that you 
appreciate whatever he/she is able to tell you about the topics you’ll be exploring. 

We have tried to indicate the kinds of follow up questions you might make in the context of the 
questions below. However, following up on evaluative statements is generally important, and 
warrants mentioning here. Whenever possible, follow up on subjects’ evaluative statements: 
e.g., S: “That was a hard class” ...Int: “What was hard about it?” or “How was it hard?”; S: “I 
like math”... Int: “What do you like about math? Can you give me a specific example?” 

Before interviewing a subject, remember to re-read and familiarize yourself with relevant data 
collected on that subject, including fieldnotes, the Year 1-3 ethnographic interviews, and any 
informal interviews that may have been conducted. Also review the current state of the analysis 
of that subject’s data. This will allow you, on a case-by-case basis, to tailor questions to a 
particular subject. This will greatly increase the comparative power of the interviews for the 
longitudinal aspect of the study. We are including a formula for constructing a question based on 
this data in the interview this year. A good place to start might be looking at what a student was 
“concerned about” when they were asked the question in the year 3 interview. 

So, find an interesting topic you would like to ask the student about based on the data collected 
from previous years focusing upon year 3 interviews or concerns students brought up in the 
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informal interviews in the Fall of ‘06. If the pieee of data you are using eomes from an old 
interview, then a natural place to ask them about this topic again would be the same question it 
came up the first time. For example, lets say in year 3 interview a subject, “Leslie” said that she 
was really worried about finding a job before graduation. When I get to question #18 this year, 
before I ask the question I will say “Leslie, last year in our interview you talked about finding a 
job before graduation, can you talk about how that has gone? 

So our formula would be. . . . 

[Last year/when I observed you...] you talked about [insert student’s topic of 
concern/excitement...], talk about how that has gone this year. 


In the protocol below we have inserted this placeholder in a few places where we believe that the 
question is important. Please feel free to use your judgment to explore potential points of 
interest that might arise based on your review of your past transcripts. Sometimes the most 
important issues have not been anticipated by the researchers at the outset. 
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Academic Pathways Study Ethnographic Interview 
Interviewer’s Guide 
February 23, 2007 


1) Tell me how things are going for you? 

a. Explore further on specifics of response, if student does not provide details: e.g., if student 
says, “It’s been a lot harder than I thought,” ask, “In what ways?” 

Note: Follow the subject’s lead on how to handle this question. In some cases, it might be 
best used as an icebreaker, just to begin the conversation and move on to other 
questions. However, some subjects (most, based on our informal interviews at LPub this 
year) will have quite a lot to say, on a range of topics. In this case, don’t be too quick to 
move off of this question - explore all of the issues the subject seems willing to get into. 

b. Before moving on to other questions, give subject a chance to say more, e.g., “What else is 
happening with you these days?” 

c. Either here or in the context of another question ask the student the question that revisits 
previous years’ interviews or field work and is specific to them. 

[Last year or this past Fall/when I observed you...] you were concerned about [insert 
student’s topic of concern/excitement...], talk about how that has gone this year. 


2) We talked about this some last year, but I’d l ik e to ask about it again: Can you tell me how 
you became interested in engineering? 


a. Explore further, if needed, with: What were some of the experiences that were important 
in getting you interested in engineering? 

b. [If they do not mention a specific person] Was there an individual who influenced or 
guided you to study engineering? 

3) Thinking about yourself before you came to [School Name] - are there things that you would 
say prepared you to succeed in engineering? 

4) What are you majoring in (or planning to major in)I 


a. How did you choose (xxx)? 
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b. What other majors did you consider? 

c. What other kinds of engineering did you consider? 

d. Are there certain fields of engineering that you just can’t see yourself going into? Why? 

4x) Did you have an internship during the last four years? Tell me about it. What did you do on 
a day-to-day basis? How has that internship related to school? 


5) Can you take me through a typical day for you here at [School Name]? 


6) I’d like to ask about your classes and other academic experiences you’ve had since you’ve 
been here. 

a) Have you had academic experiences that you would describe as particularly bad? Can 
you tell me about one of those? Again, if necessary, make sure to follow up with a 
question like: “So what was it that made that a bad experience?” 

b) Have you had academic experiences here so far that you would describe as particularly 
good? Can you tell me about one of those? If necessary, make sure to follow up with a 
question like: “So what was it that made that a good experience?” 

c) Tell me about your capstone/senior project? 


7) How would you say that you changed as both a person and student since you were a 
freshman? How are you the same? (ask them at the same time and let them respond) 


8) Let me ask you to think about the other engineering students you’ve come across here. Would 
you say that in general they are more different from you or more similar? 

a. Explore further: “How are they similar?” (or “How are they different?”) 


9) Have you had much contact with non-engineering students? Would you say that there are 
things that distinguish engineering students from students in other majors? 

a. Explore further as needed to get details. 
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10) What would you say has been the most diffieult thing here for you so far? [Students might 
mention non-academic dijficulties. Make sure to explore further to get at academic difficulties.] 

Note: this would be a good spot to use the placeholder noted above: 

[Last year/when I observed you. . .] you talked about [insert student’s topic of 
concern/excitement. . .], being difficult for you, talk about how that has gone this year? 


a. How did you handle (or how are you handling) that? 


11) What’s been easy for you here so far? 
a. Explore further if subject doesn ’t elaborate. 


12) Has anything surprised you about your engineering education? Did you believe that it would 
be different than what you experienced? 


13) If you were to advise me, as a freshman student, about what classes to take — when, in what 
combinations, and taught by whom, what would you tell me? 

a. Asides from courses is there anything else that you would advise me as far as getting 
involved with the engineering community on campus? 


14) Are there any groups that you’ve become part of since you came here to [Name of 
Institution]? 

a. Explore further for each group (or, if there are a lot, for what the subject would say are 

the “most important” ones) 

i. Can you tell me about [group]? 

Note: This may need some prompting to establish the range of places we are interested in, both 
on and off campus, both social and para-professional. Explore further to see if they have 
answered with respect to both on and off campus groups. 


b. Is there someone you met (professor, engineer, advisor) since you began school here who 
you would say has filled a mentor role for your engineering interests? 
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16) Knowing what you know now, as you look back on the time that you’ve spent here, is there 
anything that you would do differently? 


17) One of the things that our researeh team is interested in is diversity in engineering and 
engineering edueation, in terms of race, ethnieity, and gender. I’d like to ask you some questions 
related to this. 

Note: this series of questions may be another area where the placeholder will work well: 

[Last year/when I observed you. . .] you talked about [insert student’s topic of 
concern/excitement. . .[, talk about how that has gone this year. 

a (This is a question that you’ve already answered on the survey, but your survey responses 
aren’t available to me yet.) Can you tell me how you identify yourself raeially or ethnically? 


b. Are there supports or barriers, advantages or disadvantages, for you as a [ethnie 
identifieation] engineering student? 

i. How about for people of other raeial or ethnie groups? 


e. Do you think that there are differenees between the experienees of male and female 
engineering students? 


d. How has it been for you here, as a [male/female] engineering student? 


18) I want to ask you to think about the rest of the time that you’ll be spending at [Sehool 
Name]. 


a. What are you looking forward to during the rest of your time here? 

b. What are you eoneerned about? 


19. This is likely the last time we’ll interview you, so I’d l ik e you to talk about what happens for 
you after this interview. 

a) Tell us what happens next for you? This summer? This fall? Beyond? 

Note: this should be another question where the placeholder may be relevant: 

[Last year/when I observed you. . .[ you talked about [insert student’s topic of 
concern/excitement. . .[, talk about how that has gone this year. 
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Here we are trying to get the students to frame their future on their terms. If these 
topics don ’t come up probe to steer the conversation to topics including, but not 
limited to: 

i) when he/she is graduating (if graduating at all)? 

ii) Job search, how it went? Does he/she already have job?, etc. 

iii) What about grad school? 

iv) Options other than working in engineering? (teaching, starting a business) 

v) What about ten years from now? 


b) What do you imagine yourself doing on a day-to-day basis? 


i) AND, if not planning on becoming an engineer: What do you imagine engineers 
do on a day-to-day basis? 


20. Here’s a more personal question. I’ve been asking a lot of questions so far about your 
academic experiences. What else do you do in your life besides being an engineering student? 


22. If you could whisper in the ear of the people who set up your engineering program here at 
[Name of Institution], what advice would you give them about improving things? 


26. You are asked to testify in front of a congressional committee that is investigating the entire 
institution of engineering. As a [xxx] engineer, defend your discipline. 

a) Why do we need [xxx] engineers? 

b) What have [xxx] engineers done to make my life better? 


23) This is the last of our four interviews with you. I’d like you to comment on what you 
thought about this interview process. Is there anything that is relevant to this study that you 
think I’ve missed? 
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Respondent ID: 

Performance Task 

Transition from Ethnographic Interview: 

The last part of today’s session has a different format from the interview we just completed. I 
will be reading instructions and questions from a script. 

Instructions: 

At this time, I'd like to ask you to work on a short activity. This is the kind of activity that has 
many different kinds of answers. We would like you to give it your best effort. You have up to 
fifteen minutes to work on it. I will let you know when there is five minutes left, so you have an 
idea about how much time has passed. Please let me know if you are done before the fifteen 
minutes is up. Do you have any questions? OK, here is the activity. You might remember this 
activity if you were asked to do it two years ago as part of this study. (CHECK START TIME 
ON AUDIO RECORDER AND INDICATE IT IN THE BOX BEEOW. THEN HAND R THE 
EOUR-PAGE PEREORMANCE TASK PACKET.) 

Start time: 


PTQI. (AEEOW THE STUDENT TO READ AND SOEVE THE PROBEEM ON THEIR 
OWN) 

Transition: 

(CHECK END TIME ON AUDIO RECORDER, AND IE NECESSARY): 

Okay, it’s been 10 minutes now; you have 5 more minutes to solve the problem. 

Okay, it’s been 15 minutes now, please stop. 

End time: 


(IE YOU HAVE A YEAR 2 STREET CROSSING WRITTEN RESPONSE EOR R, GIVE IT 
TO THEM AND PROCEED TO PTQ2 ON THE NEXT PAGE. IE NOT, SKIP AHEAD TO 
PTQ4, WHICH IS MARKED WITH AT.) 


2007 rev. 
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Here’s a copy of your responses from back in 2005, when you first did the street crossing 
activity. 

PTQ2: Take a look at both the responses you just wrote today and your responses from two 

years ago. What similarities and differences do you notice between the two sets of 
responses? 


PTQ3: You ’ve told me a little about how your responses are similar or different. How about 

how you came up with them? Consider how you thought about the activity and how 
you came up with the responses you wrote down, both today and two years ago. What 
similarities and differences do you notice? 

(IT’S FINE IF R ALREADY BEGAN COMPARING THOUGHT PROCESSES (VS. 
COMPARING RESPONSES) IN ANSWERING THE PREVIOUS QUESTION (PTQ2). ASK 
THEM TO CONTINUE, E.G., “Do you notice any other similarities or differences in the way 
you came up with your response?”) 


T 

PTQ4: To what extent do you feel this is an engineering problem? 


PTQ4a: (IF NOT ALREADY ANSWERED) And why? 


PTQ5: What knowledge and skills helped you solve the problem? 


APS Research Processes and Procedures 
January 2009 


3D-17 



PTQ6: Where did you develop your knowledge and skills to solve the problem? 


PTQ6a: (IF NOT ALREADY ANSWERED) Please describe those experiences in more 
detail. 


PTQ7: Are there any everyday situations from your life that remind you of the situation 
described in the problem? 


Transition: 

(COEEECT PAPER) Great! Thank you. We hope that you ’ve enjoyed this activity and we want 
to make sure that you know that there are many right answers. We ’ve used it to collect 
information from engineering students across the nation to understand the types of things 
students think about. 


3D-18 


APS Research Processes and Procedures 

January 2009 



Respondent ID: 

As an engineer, you have been asked to solve a problem on the State University campus. Just 
like campuses across the country, the State University campus is often overcrowded with 
pedestrians crossing the streets. 

One busy intersection on campus is the crossing of Fifth Ave. in front of the bookstore. Dangers 
at this intersection include heavy traffic and busses which run against the general traffic flow 
(see diagram below). The University would like to design a cost effective method for students to 
cross Fifth Ave. which would reduce the possibility of accidents at this intersection. You have 
been assigned to design a solution to this problem for presentation to the University Traffic 
Committee. 


One lane 
for buses 









Parking 

Lot 


In the process of designing your solution you have been asked to respond to the set of questions 
on the following pages. The interviewer has more paper if you need it. 


2007 rev. 


APS Research Processes and Procedures 
January 2009 


3D-19 


1 - What is the problem as you see it? 


2 - List potential solution(s) for this problem. 


3 - From your list in Question 2, choose the potential solution you think is best and provide a 
detailed evaluation of your solution. 


4 - What kinds of additional information would help you solve this problem? 
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Appendix 3-E 

Workplace Interview Guides: BCC 


Interview Guide for New Engineers 
How do new engineers begin their careers and succeed at [company]? 

Background Information for Interview Process 


Focused Interview 

Help interviewee: 

• Maximize range 

• Recall past reactions to past events, not present reactions to past 

' Elicit specific reports 

events 

• Elicit depth — cognitive, 

• Link responses to past situation (concrete, not introspection) 

affective, evaluative 

• Avoid superficial responses 

' Elicit personal context 

• Focus on past situation, not interview 

through prior experiences 
and personal attributes 


Critical Incidents 

Incidents 

• Complete activity 

• Learning 

’ Clear intent and 

• Exchange 

consequences 

• Technical 

• Social 

• Task oriented 

• Group process oriented 

• Supervisor oriented 

• Organization oriented 

Developing Expertise 

How is technical (vs. social/people) defined? Wow factor 
Applying eng. knowledge and skills to practice 
Identifying most valuable knowledge/skills from school 
Perceptual changes of engineering in practice 
Patterns of continuing education and development 

Socialization 

Adjusting to fit: assimilating 
Learning (content) 

—tasks, group procedures, supervisor expectations, org culture/values 

Learning (process) informal 

—trigger, select strategy, implement, assess 

Social exchange (LMX, CMX) 

—actors, resources, exchange structure, exchange process (initiate, 
transact, reciprocate) 

—amount of interaction 

—direct, general, productive (negotiated) 

—LMX: respect, trust, responsibility, autonomy, mutual obligation 
—role clarity 

—perceptions of success, commitment, satisfaction 
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Interview Guide for New Engineers 


Introduction 

Introductory remarks 

1 . State purpose and sponsorship of study 

2. Group sampled/interviewed (answer “Why me?”) 

3. Anonymity of data (show how anonymity will be guarded) 

Overview of Study 

I am with [school] and am are working with HR at [company]. We are 
interested in improving the educational programs for engineers and 
the onboarding programs at [company]. Specifically, what helps you 
work successfully within [company] . 

Identify incidents 

Think back to when you started working here. 

What was the [first, second, third] incident or experience you had: 

Expertise 


Problem solving process 

technical 

Tasks & procedures 

in which you had to apply your engineering knowledge/skills to solve 
a problem or accomplish some task 

Socialization 


Learning process 

social 

Exchange process 

that taught you something about how things get done here? That is, 
the way things are done here and what was expected of you. 

Behaviors 


Thoughts 

[Identify 2-3 critical incidents each (technical & social)] 

Emotions 



Time 

When did this incident happen? 

Place 

Where did this incident take place? 

Antecedents 

What were the general circumstances leading up to this incident? 
Personal Context 

What were your circumstances surrounding this incident? 

Actors 

Who was involved? 

Behaviors and interactions 
What exactly did the others do? 

Behaviors and interactions 
What exactly did you do? 

Consequences 

What was the outcome of this incident? 

Motivations 

Why did you do this? 

Move to technical section T 
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Expertise: Engineering 
knowledge, skills, attributes 
(KSAs) at entry and at present: 
how changed? 

For 2-3 technical incidents 

• Problem-solving and tasks/procedures 
' Applying knowledge of engineering 

• Management of projects 

• Team work with coworkers, others, supervisor 


I assume you spend your time at work doing technical type of work, 
but also you have to work with other people and the procedures of the 
organization. First of all, I want to focus our discussion today on how 
you apply your technical knowledge to solving problems and doing 
engineering tasks. 

• Think back about a specific technical problem or task 

• What was the problem? 

How did you define/frame the problem? 

Conceive 

Design 

• What knowledge and skills did you apply to work on the 
problem? 

- Where did you learn the knowledge and skills (topic)?. 

- How did you learn the topic? Bookwork, labwork 

- Implement 

- Operate 

• What questions/concerns did you have? 

• How did you find answers? 

• What do you wish you knew then? 

• Where did you get the knowledge you were missing? 

• What did you learn from this situation? 

• What would you do differently (what did you learn from this 
experience)? 

- Operate 

Are these typical or unique events among your peers? 

Transition 

Difference between school and work 
Move to socialAearning section T 
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Socialization: Informal 
Learning and Relationship 
Building 

For 2-3 social incidents 

For each incident, discuss the learning process and learned outcomes 
related to cognitive affectual, and social 

Transition to social 

Now, I would like to talk about the social/people aspects of your 
work. Specifically, I’d like to talk about how you learned the 
[company] way to work with other people, other departments, and 
how you learn what others expect of you on the job. 

NORMS 

Cognitive knowledge 
Behavior skills 

About a specific incident: 

Describe what you expected to occur/ought to occur (previous 
knowledge, mental model) 

Actually occurred 

Describe how you handled the experience. 

Describe what you learned from that incident, [specific knowledge, 
skills, meanings, understandings] 

[Specify cognitions in concrete details: range, specificity, depth, 
personal context] 

Do you think this is a typical/common experience? 

feelings, emotions, moods, 
motivations 

Describe how you felt about that incident, [specific feelings, emotions, 
moods, motivations] 

[Specify feelings, emotion, mood, motivation in concrete details: 
range, specificity, depth, personal context] 

Social participation, 
communication, cooperation 

Describe how you interacted with others before, during, and after the 
incident. 

[Specify participation, communication, and cooperation in concrete 
details: range, specificity, depth, personal context] 

Social Exchange supervisor, 
coworkers 

respect, trust, and obligation 

' Describe your relationship with your supervisor. 

• Describe your relationship with your coworkers. 

• Probe for indications of respect (engagement in decision- 
making), trust (degree of autonomy and responsibility), and 
mutual obligations (commitment) 

Understanding of success 

What do you think will help you and/or others succeed here? 
What could [company] do to help new engineers succeed? 

What do you see yourself doing in 3-5 years? 

Closing 

Any other comments? 

Advice? For new hires. For school? 
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Interview Guide for Supervisors 
How do new engineers begin their careers and succeed? 

Background Information for Interview Process 


Focused Interview 

Help interviewee: 

• Maximize range 

• Recall past reactions to past events, not present reactions to past 

- Elicit specific reports 

events 

• Elicit depth — cognitive, 

• Link responses to past situation (concrete, not introspection) 

affective, evaluative 

• Avoid superficial responses 

• Elicit personal context 

• Focus on past situation, not interview 

through prior experiences 
and personal attributes 


Critical Incidents 

Incidents 

• Complete activity 

• Learning 

’ Clear intent and 

• Exchange 

consequences 

• Technical 

• Social 

• Task oriented 

• Group process oriented 

• Supervisor oriented 

• Organization oriented 

Developing Expertise 

How is technical (vs. social/people) defined? Wow factor 
Applying eng. knowledge and skills to practice 
Identifying most valuable knowledge/skills from school 
Perceptual changes of engineering in practice 
Patterns of continuing education and development 

Socialization 

Adjusting to fit: assimilating 
Learning (content) 

—tasks, group procedures, supervisor expectations, org culture/values 

Learning (process) informal 

—trigger, select strategy, implement, assess 

Social exchange (LMX, CMX) 

—actors, resources, exchange structure, exchange process (initiate, 
transact, reciprocate) 

—amount of interaction 

—direct, general, productive (negotiated) 

—LMX: respect, trust, responsibility, autonomy, mutual obligation 
—role clarity 

—perceptions of success, commitment, satisfaction 
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Interview Guide for Supervisors 


Introduction 

Introductory remarks 

4. State purpose and sponsorship of study 

5. Group sampled/interviewed (answer “Why me?”) 

6. Anonymity of data (show how anonymity will be guarded) 

Overview of Study 

I am with [school] and am are working with HR at [company]. Our 
interest has to do with how new engineering graduates make the 
transition from school to work. Specifically, what helps them work 
successfully within [company] . 

Identify roles 

What are your roles as a supervisor related to supervising a new 
engineer(s)? 

Identify incidents 

- Most important 

- Least important 

Expertise 


Problem solving process 

Think back (over the past year) about a specific incident in which you 

Tasks & procedures 

interacted as a supervisor with a new engineer. 

Socialization 

What was the [first, second, third] incident or experience you had; 

Learning process 


Exchange process 

Describe the incident: 

- Context, antecedents 

Behaviors 

- Roles 

Thoughts 

- Behavior 

Emotions 

- Consequences 

- Relationship factors 

- Exchange factors 

- Expectations 

Time 

When did this incident happen? 

Place 

Where did this incident take place? 

Antecedents 

What were the general circumstances leading up to this incident? 
Personal Context 

What were your circumstances surrounding this incident? 

Actors 

Who was involved? 

Behaviors and interactions 
What exactly did the others do? 

Behaviors and interactions 
What exactly did you do? 

Consequences 

What was the outcome of this incident? 

Motivations 

Why did you do this? 

Move to learning section T 
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Expertise: Engineering 
knowledge, skills, attributes 
(KSAs) at entry and at present: 
how changed? 

For 2-3 technical incidents 

• Problem-solving and tasks/procedures 
' Applying knowledge of engineering 

• Management of projects 

• Team work with coworkers, others, supervisor 


First of all, I want to focus our discussion on your observations about 
how new engineers apply their technical knowledge to solving 
problems and doing engineering tasks. 

• What skills do you look for in a new hire? 

• What important technical skills do you think new hires often 
lack? 

• Any characteristics of new hires in the way they solve problems/ 
work on projects? 

• What knowledge and skills did you think should apply to work on 
the problem? 

• What questions/concerns do you typically have with new hires? 

• How do you become comfortable with their abilities? 

• How do help them learn the details of their job? 
mentors 

senior eng. 

• Do you see any differences between new grads and experienced 
new hires? What are they? 

• Anything you would do differently? 

Are these typical or unique events among your peers? 

Move to socialAearning section A 
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Socialization: Learning & 
Relationship Building 


Role 

Describe what you see as your role as a manager/supervisor. 

Describe how you helped the new hire learn what to know and the 
proper way to do things. 

Describe what you have learned over time about managing new 
people, [specific knowledge, skills, meanings, understandings] 

Culture 

Describe important characteristics of the culture of the company. 
[Artifacts, values, assumptions] 

What are some of the key values and assumptions of the company? 

Norms 

Social participation, 
communication, cooperation 

Describe what you expected the new employee to know and do. 

- Attitudes, Knowledge, Skills, Behavior 

Describe how you interact with those that report to you. 

[Specify participation, communication, and cooperation in concrete 
details: range, specificity, depth, personal context] 

How do you foster engagement with new hires? 

How do you integrate new hires into the company? 

Social Exchange supervisor, 
coworkers 

respect, trust, and obligation 

• Describe how you develop productive relationships with new 
employees. 

• How do you develop trust, autonomy, and commitment with 
employee? 

• What are the stages of development for new employees? [first 
year; first 5 years] 

• How/what kind of help from others? 

• Probe for indications of respect (engagement in decision- 
making), trust (degree of autonomy and responsibility), and 
mutual obligations (commitment) 

Understanding of success 
Perceptions of commitment 

• What do you think helps new employees succeed here? 

• What are the attributes of stars/quick starters? 

• What can you/[company] do to foster success? 

Closing 

Any other comments? 
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Appendix 3-F 

Workplace Interview Guide: County-State-Aerospace 


• Tell me how things are going for you? 

• How has your work as a engineer met your expectations? 

• How was your education prepared you for the kinds of work that you are doing now? 

How so? How not? 

• Based upon your experience working as an engineer are there any classes you wish 

you had taken or worked harder in? 

• What about teamwork? 

• Take me through a typical day at your workplace? 

• Describe some things that you learned in your workplace that you never imagined being 
important to work as an engineer. Has there been anything you encountered in the 
workplace that you never imagined being a part of engineering? 

• How would you say that you have changed as a person and as an engineer since you 
graduated? 

• Describe some of your co-workers? How are you similar and/or different than them? 
Were these similarities and differences surprising? Do you find there is a level of 
competition in the workplace? 

• What has been easy for you thus far in workplace? What have been some difficulties you 
have encountered? 

• If you could whisper in the ear of the people who set up your engineering program here at 
[Name of Institution], what advice would you give them about improving things? 

• Where do you see yourself in five to ten years? What’s next for you? Grad school? 
Staying with your current job? 
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Appendix 3-G 

APS Longitudinal Cohort Exit Interview Guides 


Academic Pathways Study 
Exit Interview 

Technical Public Institution/Urban Private University/ 

Suburban Private University/Large Public University 
2005-2006 

1 . Can you tell me about your reasons for majoring in engineering when you started at 
(name of school)? 

Note: For this question in particular, it will be worth knowing how a given 
subject has answered this question (or similar questions) previously. We 
wouldn ’t suggest being confrontational ( i.e., “But wait, that’s not what you said 
last year... ’’), but previous responses can usefully inform follow-ups for this 
question. 

This can be a very useful question analytically, in that it can be compared to what 
students said in their Year 1 interviews. We expect that students might answer 
this question retrospectively, from the perspective of someone having left. 

Please probe the answer to gauge whether the student had a prior connection 
to engineering through family or friends or some other type of exposure. Did 
the student’s understanding of what engineering would be like change? 
Question #4, below, gets at this as well. 


2. Think about your expectations when you started last year. Did things go as you 
expected? What did? What didn’t? 

3. What’s your current major? How did you choose XXX? How is it going? 

4. a. When did you first start to think about switching out of engineering? (or, if more 

appropriate, into their current major - some students might frame leaving 
engineering as a positive switch into something that was more appealing to them.) 

Follow up with, e.g., “What was going on at the time to make you start 
considering this?, ” and explore each of the things they discuss. 

4.b. So from that point, can you talk me through the process until now? 

Follow up with, e.g., “What was switching like for you? Were there difficult 
moments ? Were there moments when you doubted the direction you were headed 
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in ? Were there moments when you were certain you were headed in the right 
direction ? Were there any particularly important events/moments/etc. ? ” Any 
specifics should be explored further for more detail. 

5. a. I’d l ik e to ask about some of your specific experiences. Let’s start with your 
classes. Did you have any classes that you would say were particularly good ones? 
(Follow up on why they were good, looking for details.) Did you have any classes that 
you would say were particularly bad ones? (Follow up on why they were good, looking 
for details.) 


5.b. How about out of class? Any particularly good/bad experiences? (Follow up as 
above) 


Note: If student volunteers non-academic out of class experiences, let them talk 
about those. We’d suggest then directing them back to out of class academic 
experiences (e.g., tutoring, office hours, etc.) The mentor questions might come 
up here as well. 


6. How would you say you’re feeling at this point about leaving engineering? 

7. What’s different for you now that you’re (no longer an engineering major/ an xxx 
major)? 

8. What’s ahead for you? What are you planning for when you leave (name of school)? 

9. If you could talk to the people who run the engineering program, what would you tell 
them? 

10. Do you have anything to add to what you’ve said so far? 
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Academic Pathways Study 
Exit Interview 
Large Public University 
2005-2006 

1 . Can you tell me about your reasons for intending to major in engineering when you 
started at LPub? 

la. Were you aware of the engineering admission process? 

Note: For this question in particular, it will be worth knowing how a given 
subject has answered this question (or similar questions) previously. We 
wouldn’t suggest being confrontational (i.e., “But wait, that’s not what you said 
last year... ’’), but previous responses can usefully inform follow-ups for this 
question. 

This can be a very useful question analytically, in that it can be compared to what 
students said in their Year 1 interviews. We expect that students might answer 
this question retrospectively, from the perspective of someone having left. 

Please probe the answer to gauge whether the student had a prior connection 
to engineering through family or friends or some other type of exposure. Did 
the student’s understanding of what engineering would be like change? 
Question #4, below, gets at this as well. 


2. Think about your expectations when you started last year. Did things go as you 
expected? What did? What didn’t? 

3. What’s your current major? How did you choose XXX? How is it going? 

4. a. When did you first start to think about switching out of engineering? (or, if more 

appropriate, into their current major - some students might frame leaving 
engineering as a positive switch into something that was more appealing to them.) 

Follow up with, e.g., “What was going on at the time to make you start 
considering this?, ’’ and explore each of the things they discuss. 

4.b. So from that point, can you talk me through the process until now? 

Follow up with, e.g., “What was switching like for you? Were there difficult 
moments ? Were there moments when you doubted the direction you were headed 
in ? Were there moments when you were certain you were headed in the right 
direction ? Were there any particularly important events/moments/etc. ? ’’ Any 
specifics should be explored further for more detail. 
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5. a. I’d like to ask about some of your specific experiences. Let’s start with your 
classes. Did you have any classes that you would say were particularly good ones? 
(Follow up on why they were good, looking for details.) Did you have any classes that 
you would say were particularly bad ones? (Follow up on why they were good, looking 
for details.) 


5.b. How about out of class? Any particularly good/bad experiences? (Follow up as 
above) 


Note: If student volunteers non-academic out of class experiences, let them talk 
about those. We’d suggest then directing them back to out of class academic 
experiences (e.g., tutoring, office hours, etc.) The mentor questions might come 
up here as well. 


6. How would you say you’re feeling at this point about leaving engineering? 

7. What’s different for you now that you’re (no longer an engineering major/ an xxx 
major)? 

8. What’s ahead for you? What are you planning for when you leave (name of school)? 

9. If you could talk to the people who run the engineering program, what would you tell 
them? 

10. Do you have anything to add to what you’ve said so far? 
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Appendix 4-A 

Longitudinal Cohort (PIE) Survey Examples 


Survey 1, Winter 2004 4A-2 

Survey 4, Spring 2005 4A-30 

Survey 7, Spring 2007 4A-42 
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Academic Pathways Study Winter'04 Survey 

Please click the SUBMIT button only after you have completed the survey. You 
will then need to enter your login and password. For best viewing results, please 
maximize your browser window. 

1 . What is your expected year of graduation from coiiege? 

^ n2003 

t" n2004 
02005 
^ 02006 
<" t" 02007 
("n2008 
'"□2009 
<"02010 or later 

2. What do you intend to major in? (Currentiy, your first choice) 

<~ <" OAeronautical Engineering 

^ <" OChemical Engineering 

<" <" OCivil and Environmental Engineering 

<" OComputer Science 
<~ <" OEIectrical Engineering 

^ <" OManagement Science and Engineering 
- r DMaterials Science and Engineering 
r oMechanical Engineering 
^ <" OOther Engineering 

<"oArts and Humanities 
("oEducation 

' <" OPhysical Science/Math 

<" OSocial Science 
<" <" OOther: 

3. What do you intend to major in? (Currentiy, your second choice) 

<~ (''OAeronautical Engineering 
(" OChemical Engineering 
(" (" OCivil and Environmental Engineering 
' ^ (" OComputer Science 

("OEIectrical Engineering 
0 r r OManagement Science and Engineering 
0 r r DMaterials Science and Engineering 
-.'C ("oMechanical Engineering 
( C ("oOther Engineering 
( C TnArts and Humanities 
j r r oEducation 
j r r nPhysical Science/Math 
c r nSocial Science 
'■(' ("on/ A 
C Tnother: 

4. What is the highest academic degree that you intend to obtain in engineering? (Mark one) 

("oNone 

-C rnBachelor’s 
J C r DMaster’s 
jc rnPh.D. 

•0 c r nother 
' C Tqi do not know 
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5 . If you are thinking of going to graduate school (NOT in engineering), please mark your probable area of study. 

^ DBusiness 
^ <" DEducation 

^ DMedicine 
nLaw 

TnMA/Ph.D. 

("nPublic Service 
^ <" DOther 
^ <" DN/A 

6. About how many hours do you spend in a typical 7-day week doing each of the following? 

0 1-5 6-10 11-15 16-20 21-25 26-30 more than 30 

Preparing for ciass (studying, reading, writing, doing homework or iab work, anaiyzing data. 


rehearsing, and other academic activities) 

r rn r rn r r t rn t 

□ 

r 

C n 

jr 

'^n 

Working for pay on campus 

r rn r rn r rn ./r rn jr rn jr 

□ 

jr 

n 

jr 

□ 

Working for pay off campus 

r m r rn r rn r rn r rn r 

D 

r 

□ 

r 

^ D 

Participating in co-curricuiar activities (organizations, campus pubiications, student government, 

sociai fraternity or sorority, intercoiiegiate or intramurai sports, etc.) 

<" <"□ ^ D 

D 

jr 

□ 

r 

D 

Reiaxing and sociaiizing (watching TV, partying, exercising, etc.) 

r rn r rn r rn r rn r rn r 

0 

r 

n 

r 

r 

Providing care for dependents iiving with you (parents, chiidren, spouse, etc.) 

r rn r rn ^ n rn , r rn r 


jr 

□ 

Commuting to ciass (driving, waiking, etc.) 

r m r rn r m r Tn jc rn jr 

D 

jr 


jr 

r 


7 . For each reason for studying engineering, please indicate how strongly you disagree or agree with the statement: 
Strongly Disagree Disagree Neutral Agree Strongly Agree 
I enjoy figuring out how things work 

Technoiogy piays an important roie in soiving society’s probiems 

r To r rn r Tq r To r . 

Engineers make more money than most other professionais 


^ rn r rn 

r 

rn 

r 

n 

, r 

r n 

My parent(s) are making me study engineering 

r rn r rn r rn r 

r n 

jr 

r n 

Engineers are innovative 

r rn r rn 

r 

r n 

r 

D 

, r 

r n 


Engineers have contributed greatiy to fixing probiems in the worid 

r rn r (•'□ m r ,-r > 

Engineers are weii paid 

^ D ^ ^ (~n n d 

Engineering is an occupation that is respected by other peopie 


r rn r 

rn r 

rn 

r 

r n 

r 

r 

My parents want me to be an engineer 





r rn r 

rn r 

r n 

r 

r n 

r 

r n 

An engineering degree wiii guarantee me a job when 1 graduate 

r rn r 

rn r 

r n 

r 

r n 

r 


Engineers are creative 







r m . r 

rn jr 

r n 

jr 

r n 

.^r 

D 
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8. For each statement, please Indicate how strongly you disagree or agree with the statement: 

Strongly Disagree Disagree Neutral Agree Strongly Agree 

I prefer working/studying alone 

r r rn r To t rn t Tq 

I enjoy the subjects of science and mathematics the most 

r rn r m r rn rn t r 

Creative thinking is one of my strengths 

r rn r n <" rn ‘r rn /r rn 

Studying in a group is better than studying by myself 

r c'n ^ D cn r m -jr rn 

I have strong problem solving skills 

n ^ ("n r rn rn r rn 

I enjoy taking liberal arts courses more than science and math courses 
r rn r rn r rn r rn . r - 

I enjoy problems that can be solved In different ways 

^ ("n ^ rn r rn rn r rn 

My Instructors often remind students that they need to do better than other students to obtain high 
grades 

r rn r rn r rn r ^'n r . 

I have easy access to work spaces where I can participate In peer study/dlscusslon sessions with 
my fellow students 

r rn r rn r rn -.r rn r rn 

I am encouraged by my Instructors to Initiate orparticipate In peer study sessions with my fellow 
students 

r rn -r rn -C rn .-T rn jr rn 

My Instructors grade on a curve. 

r rn --r rn --r rn rr rn rr rn 


9 . Please Indicate the Importance to you personally of each of the following: 

Not Important Somewhat Important Very Important Essential 

Getting higher grades than my classmates 

r rn r rn r rn r rn 

Influencing social values 

r rn r rn -c rn .-r rn 

Becoming an authority In my field 

r rn r rn r rn r rn 

Keeping good Ideas to myself unless It Is to my advantage to share them 

r rn r rn r rn r : 

Helping to promote racial understanding 

r rn r rn r rn r rn 

Becoming a community leader 

r rn r rn .r rn -r rn 

Helping others who are In difficulty 

r rn r rn r rn rn 

When playing any game, to win 

r r'n r r"n r rn -c rn 

Developing a meaningful philosophy of life 

r rn r rn r rn r rn 

Becoming a practicing engineer 

r r'n r r'n ...r rn -'G rn 

Getting along with others 

r rn r rn rn jr rn 

Working as part of a team 

r m r rn rr rn 'r rn 

Becoming a student government official 

r r'n r rn r rn r rn 

Establishing relationships with engineering students 
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r To r Tq r 

Establishing relationships with non-engineering students 
r To r rn r rn r . ■ 


10. Compared with when you entered this coiiege, how wouid you describe your: 

Much Weaker Weaker No Change Stronger Much Stronger 

Analytical and problem solving skills 

r c'Q r rn r m m 

Critical thinking skills 

r To c c u jc Cn jc r □ jr Cu 
General knowledge 

r "'□ r rn -r rn r rn jr rn 

Knowledge of a particular field or discipline 

^ D f'n 

Interest in studying engineering 

r rn r rn r rn jr rn jr rn 


11 . Piease rate your satisfaction with this institution on each of the aspects of campus iife listed below. If you do not have 
experience with this aspect, mark "n/a." 


Very dissatisfied Dissatisfied 

Neutral 

Satisfied 


Very satisfied 

Opportunities for community service 

r rn r rn r rn 


^ n 

^r 

r n 

^r 

r n 

Quality of instruction by faculty 

r rn r m -r rn 

jr 

rn 

jr 

rn 

jr 

rn 

Availability of faculty 

r rn r m -r rn 

jr 

r n 

jr 

r n 

jr 

r n 

Quality of instruction by teaching assistants 

r ^'n ('n ■<" 

jr 

r n 

jr 

r n 

jr 

r 

Availability of TAs 

r rn -r rn jr rn 

jr 

r n 

. r 

r n 

jr 

r n 


12. Piease rate your satisfaction with each of the foiiowing at this institution, if you do not use the service or faciiity, mark "n/a." 


Very dissatisfied 


Dissatisfied 


Neutral 

Satisfied 


Very satisfied 

Computer facilities 
r rn 

r 

r n 


rn 


rn 


rn 


rn 

Libraries 

;r m 

•r 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

Classrooms 

r rn 

r 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

Tutoring 

r r: 

r 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

Academic advising 

^ rn 

r 

r n 

jr 

r n 

jr 

r n 

jr 

r n 

jr 

rn 

Laboratories 

r Cq . 

:r 

r n 

jr 

r n 

- - r 

r n 

r 

r n 

jr 

r n 


13 . Since entering this coiiege, indicate how often you: (Mark one for each item) 

Not at all Rarely Occasionally Frequently 

Felt that your courses inspired you to think in new ways 

r r rn c r^ c c'n 

Felt you did not have enough time to pursue nonacademic activities 

r rn r rn r rn r r 

Worried about keeping up with your schoolwork 

r rn r rn r rn r rn 

Felt you did not have a "social life" 

r f'n o c'n c r^ r rn 

Worried about how you would pay for school 

r rn r rn r rn t rn 

Felt stressed 
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r r- r T Tq -_.r Tn 

Participated in a peer study group 

r rn r c-'n r Cu To 


14 . Think about the math classes you have taken. Since entering college, indicate how often you: (Mark N/A if you have not 
taken any math classes) 

Not at all Rarely Occasionally Frequently N/A 

Came late to math class 


r rn 

r rn 

,jr rn rr 

Skipped math class 



r rn 

r rn 

-r rn ^r 


Turned in math assignments that did not reflect your best work 

r rn r rn r rn rn r 

Turned in math assignments late 

r rn r rn r rn r rn -..r rn 

Thought math classes were boring 

r rn rn --'C rn rr rn .jc rn 


15 . Think about the science classes you have taken. Since entering college, indicate how often you: (Mark N/A if you have nof 
taken any science classes) 

Not at all Rarely Occasionally Frequently N/A 

Came late to science class 

r rn r rn -jr rn rn jr rn 

Skipped science class 

r rn r rn -jr rn .-T rn jr rn 

Turned in science assignments that did not reflect your best work 

r rn r rn r rn .-'O rn jt r 

Turned in science assignments late 

r rn r rn rn r rn ...r rn 

Thought science classes were boring 

r rn - r rn - r rn rr rn jr r n 


16 . Think about the engineering classes you have taken. Since entering college, indicate how often you: (Mark N/A if you have 
not taken any engineering classes) 

Not at all Rarely Occasionally Frequently N/A 

Came late to engineering class 

r f'n r rn .-'O rn .-'0 rn jc rn 

Skipped engineering class 

r rn r rn ,:r rn -r rn 'O rn 

Turned in engineering assignments that did not reflect your best work 

r rn r m r rn ...r m ..-r r 

Turned in engineering assignments late 

r rn r rn r rn -'T rn rn 

Thought engineering classes were boring 

r m r rn r rn jt Tq /r rn 


17 . Think about the liberal arts (not math, science or engineering) classes you have taken. Since entering college, indicate how 
often you: (Mark N/A if you have not taken any math classes) 

Not at all Rarely Occasionally Frequently N/A 

Came late to liberal arts class 

r f'n r rn ,.r rn rr rn jr rn 

Skipped liberal arts class 

r rn r rn r rn r rn r rn 

Turned in liberal arts assignments that did not reflect your best work 

r rn r rn r m r rn r / 

Turned in liberal arts assignments late 

r rn r rn r c'n rn rn 

Thought liberal arts classes were boring 

r rn r rn r r'n --*r rn jr rn 
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18 . Since entering this coiiege, how often have you interacted with the foiiowing peopie (e.g. by phone, e-maii, instant 
Messenger, or in person)? (Mark one for each item) 


Never 1 -2 times/term 1 -2 times/month 


Once a week 

2-3 times/week 

Daily 

Faculty during office hours 

r rn r rn r rn 

jr 

^ D 

jr rn 

jr rn 


Faculty outside of class or office hours 

<"<"□ F~rn 


C D 

or rn 

jr rn 


Teaching Assistants during office hours 

r r'n r c'n r rn 

r 

^ D 

jr rn 

jr rn 



Teaching Assistants outside of ciass or office hours 

("n r r rn r rn jr Tq jr ^ 


19 . Rate yourseif on each of the foiiowing traits as compared with the average person your age. We want the most accurate 
estimate of how you see yourseif. (Mark one in each row.) 

Lowest 10% Below Average Average Above Average Highest 10% 


Self confidence (intellectual) 

r , r f-'n 

r 

r n 

^r 

rt 


r n 

Self confidence (social) 

r rn rn 

jr 

rn 

jr 

r n 

or 

r n 

Self understanding 

r ^'n jr rn 

jr 

r n 

or 

r n 

or 

r n 

Leadership ability 

r f'n r rn 

jr 

rn 

r 

r n 

or 

rn 

Public speaking ability 

r >' r rn 

jr 

r n 

or 

r n 

or 

r n 

Math ability 

r r n - ’ r r n 

’r 

r n 

r 

r n 

or 

r n 

Science ability 

r r rn 

r 

r n 

or 

r n 

or 

r n 

Computer and programming skills 

r rn r rn r 

r n 

r 

r n 

or 

r n 

Written communication skills 
r r rn 

jr 

r n 

or 

r n 

or 

r n 

Business ability 

r rn -r rn 

jr 

r n 

or 

r n 

or 

r n 


20. How important do you think each of the foiiowing skiiis and abiiities is to becoming a successful engineer? (Mark one in 
each row.) 


Not Important 

Somewhat Important 

Public speaking ability 




r rn 

r 

r n 

_r 

r n 

Math ability 





r rn 

or 

r n 

or 

r n 

Science ability 





r rn 

■ r 

r n 

r 

r n 

Computer and programming skills 


r ^n 

r 

r n 

r 

r n 

Written communication skills 



r rn 

■r 

r n 

r 

r n 

Business ability 





r ^n 

)r 

rn 

or 

r n 


Important Very Important Crucial 


_r 

r n 

^ r 

r n 

or 

r n 

or 

rn 

or 

r n 

or 

r n 

or 

r n 

or 

r n 

or 

r n 

or 

r n 

or 

r n 

or 

r n 


21 . What percentage of the courses you have taken thus far have been taught primariiy by graduate students? 

nNone 

^ nVery little 

DLess than half 
DAbout half 
^ DMore than half 
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^ ("nAII or nearly all 


22. What portion of the courses you have taken so far have used the following teaching methods? 

None Very little Less than half About half More than half 
Lectures 


r rn 

-:c 

rn 

_ r 

□ 

_ r 

D 

_ r 

^ □ 


^ D 

Individual Projects 
r tq 

.)C 

rn 

jr 

^ n 

jr 

rn 

jr 

(' D 

jr 

Td 

Team Projects 

r rn 

jr 

^ n 

JC 

C' □ 

jr 

C D 

jr 

□ 

jr 

D 

Labs 

Cu 

jr 

r □ 

jr 

□ 

jr 

^ □ 

jr 

n 

jr 

□ 

Seminars 

r Cu 

jr 

□ 

jr 

^ □ 

jr 

^ □ 

jr 

^ □ 

jr 

^ n 


23. Do you see your self studying or practicing engineering next year? 

DYes 

DNo 

<"□! do not know 

24. Your sex: 

^ DMale 
^ <" DFemale 

25. How old will you be on December 31st of this year? (Mark one) 

<"nl 6 or younger 
mi 
r rni8 
'<" <"d19 
("n20 
<^D21-24 
F" r D25-29 
D30-39 
r r 040-54 
^ DSS or older 

26. Please Indicate your ethnic background: (Mark all that apply) 

r r White/Caucasian 
r n DAfrican American/Black 
r n □American Indian/Alaska Native 
r n nAsian American/Asian 
r n DNative Hawaiian/Pacific Islander 
- H n DMexican American/Chicano 
r n DPuerto Rican 
r n nOther Latino 
r r nOther 

27. Citizenship Status: 

(''□U.S. Citizen 

^ ^Permanent Resident (green card) 

^ (^nNeither 

28. What is your marital status? 

<”□ Not married 
^ ^ DMarried 
^ nDivorced 

^ ^ nSeparated 
Widowed 

29. How many dependents do you have? 

^ ^ nNone 


All or nearly all 
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?r r ni 
'T r U2 
-r Cn3 
r rn4 

<^D5 or more 

30 . What year did you graduate from high school? 

31 . How would you describe the community where you attended high school? 

^ DRural 
nSmall town 
^ nSuburban 
^ DUrban 

32 . What was your average grade In high school? (Mark one) 

^ (^nAorA+ 

<"nA 

- ^ DB+Li 

Tdb 
^ ^db 
rc+ 

^ ("nc 

^ ^DD 

33 . Do any of your Immediate family members hold an engineering degree? (Mark all that apply) 

r TnNo 

— r n nYes, both parents 
r n DYes, father only 
r~ n DYes, mother only 
r n DYes, slbllng(s) 

34 . What Is the highest level of education that your mother completed? (Mark one box) 

DDId not finish high school 
^ <" DGraduated from high school 

DAttended college but did not complete degree 
^ nCompleted an Associate's degree (A. A., A.S., etc.) 

^ nCompleted a Bachelor's degree (B.A., B.S., etc.) 

^ (" nCompleted a Master's degree (M.A., M.S., etc.) 

^ nCompleted a Professional degree (J.D., M.D., etc.) 
nCompleted a Doctoral degree (Ph.D.) 

35 . What Is the highest level of education that your father completed? (Mark one box) 

^ nDId not finish high school 

^ <" nGraduated from high school 

<" nAttended college but did not complete degree 
nCompleted an Associate's degree (A. A., A.S., etc.) 

^ nCompleted a Bachelor's degree (B.A., B.S., etc.) 

^ (" nCompleted a Master's degree (M.A., M.S., etc.) 

<" nCompleted a Professional degree (J.D., M.D., etc.) 

^ <" nCompleted a Doctoral degree (Ph.D.) 

36 . What is your best estimate of your parents' total income last year? Consider Income from all sources before taxes. (Mark 
one) 

^ nLess than $1 0,000 

f^n$1 0,000-1 4,999 
<"n$1 5,000-1 9,999 
<^n$20,000-24,999 
n$25,000-29,999 
n$30,000-39,999 
n$40,000-49,999 
("n$50,000-59,999 
n$60,000-74,999 
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f^n$75, 000-99, 999 
<" f^n$1 00,000-1 49,999 
<~D$1 50,000-1 99,999 
<" <^D$200,000-249,999 
n$250,000 or more 


37. How do you meet your college expenses? FIN In the response that best approximates the amount of support from each of 

the various 

sources. 

None Very little Less than half About half More than half All or nearly all 


Self (job, savings, etc.) 


r rn 

r 

^ □ 

_;r 

F D 

^F 

F n 


F D 

^F 

F n 

Parents 

^ To 

)C 

F D 

jr 

F D 

r 

F D 

, r 

F D 

jr 

F D 

Spouse or partner 

^ To 

jr 

F □ 

jr 

F D 

. r 

F n 

r 

F D 

jF 

Fq 

Employer support 

^ rn 

r 

F D 

jr 

Fd 

r 

F D 

, r 

Fd 

jr 

Fd 

Scholarships and grants 
r On r 

F □ 

jr 

F D 

.■ r 

F n 

r 

F D 

jF 

F D 

Loans 

r Cn 

JC 

F D 

jr 

F D 

_r 

F D 


F D 

iC 

F D 

Other sources 

r Fn 

jr 

F □ 

jr 

F D 

JF 

F n 

jr 

F D 

r 

F Q 


38. Do you have any concern about your ability to finance your college education? (Mark one) 

^ DNone (I am confident that I will have sufficient funds) 

^ nSome (but I probably will have enough funds) 

^ nMajor (not sure I will have enough funds to complete college) 

39. Which of the following best describes where you are living now while attending college? 

DDormltory or other campus housing (not fraternity/sororlty house) 

^ <" nResIdence (house, apartment, etc.) within walking distance of the Institution 
^ nResIdence (house, apartment, etc.) within driving distance of the Institution 
DFraternlty or sorority house 

40. with whom do you live during the school year? (Mark all that apply) 

r n nNo one, 1 live alone 
r n none or more other students 
r n DMy spouse or partner 
r n DMy child or children 
r n DMy parents 

— r TnOther relatives 

r n nFrlends who are not students at the Institution 1 am attending 
r n DOther people 

41 . Do you have any of the following physical, learning, or health disabilities? (Mark all that apply) 

r n DMoblllty Impaired 

— r n DBIInd or visually Impaired 
r F DDeaf or hard-of-hearIng 
r r DOther: 

42. Do you receive any of the following assistive technology or other accommodations from the disability support services 
office on campus? (Mark all that apply) 

r I nTutorIng 

r n DTape recording of lectures 

— r F DExtended time on tests 
F F DOther: 


43. In your SCHOOL-RELATED ACTIVITIES, how often do you use the following technologies? (Mark one for each Item.) 
Never Sometimes Often Very frequently N/A 
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Desktop computer 


r rn 'r Tq jr 

G □ 

jr 

G n 

jG 

G n 

Laptop computer 

r rn -;r rn jr 

G n 

jr 

G n 

JG 

G n 

Web Browser 

r rn jr rn jr 

G □ 

jG 

G n 

JG 

G n 

Email 

r r'Q -r rn -r 

G n 

jG 

G n 

JG 

G n 

Instant Messaging (IM) Programs 

r m . r rn -r 

G □ 

jG 

G n 

JG 

G n 

Cell Phone 

r m .-r rn -r 

G n 

jG 

G n 

JG 

Gn 

Text Messaging on a Cell Phone 

r m r rn r 

G □ 

r 

G n 

r 

G n 

Personal Digital Assistant (PDA) (e.g. 

, Palm, Blackberry) 


r rn r rn r 

G n 

r 

G n 

r 

r 

Game Console 

r -n 

G n 

„G 

G n 

„G 

Gn 

Computer Games / Simulations 

r rn r rn r 

G n 

jG 

G n 

jG 

G n 

Scientific or Graphing Calculator 

r rn r rn r 

G n 

JG 

G n 

JG 

G n 

General-purpose Calculator 

r rn r m -r 

G n 

r 

G n 

JG 

G n 

Portable Music Players (e.g., CD or MP3 player) 
r rn r rn r rn r 

G n 

JG 

Gn 

Audio Recording Device 

r rn r rn t 

G n 

jr 

G n 

JG 

G n 

video conferencing or Web-Cam 

r c'n • G rn g 

G n 

jG 

G n 

JG 

G n 

Digital Camera 

r rn jG r n jg 

G n 

JG 

G n 

JG 

G n 

Chat Rooms 

r rn -r rn >r 

G n 

.tr 

G n 

JG 

G n 

Engineering Specific Software 

G C'n G rn r 

G n 

r 

G n 

JG 

G n 

General Purpose Software (e.g., Officeappllcatlons) 
GGn Grn rrn rrn 

JG 

r 


44 . In your PERSONAL LIFE outside of school, how often do you use the following technologies? (Mark one for each item.) 

Never Sometimes Often Very frequently N/A 

Desktop computer 


r rn .-r m jg 
Laptop computer 

Gn 

jG 

G n 

. G 

Gn 

r rn jg Cu jg 
Web Browser 

G n 

jG 

G n 

JG 

G n 

r m JG rn jG 

Email 

G n 

■G 

G n 

JG 

G n 

G Gn r rn r 
Instant Messaging (IM) Programs 

G n 

G 

G n 

jG 

G n 

G rn - G rn . r 

Cell Phone 

G n 

„'G 

G n 

JG 

G n 

G Gn G rn jr 
Text Messaging on a Cell Phone 

G n 

jG 

G n 

JG 

G n 

G rn G rn g 

G n 

G 

G n 

G 

G n 

Personal Digital Assistant (PDA) (e.g. 

, Palm, Blackberry) 


G ^'n -'G rn r 

Game Console 

G n 

G 

G n 

, G 

r 

G rn G rn jg 
Computer Games / Simulations 

G n 

jG 

G n 

JG 

G n 
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r -r rn Cn Cn jr To 

Scientific or Graphing Caicuiator 

r rn r jr Tq r rn jr Tn 

Generai-purpose Caicuiator 

r Tn r rn Cu 'C rn jr rn 

Portabie Music Piayers (e.g., CD or MP3 piayer) 

r rn r r'n r r-g r rn jr rn 

Audio Recording Device 

r rn r rn jr rn jr rn jr rn 

Video conferencing or Web-Cam 

r r-n .r rn r r^ jr rn jr rn 

Digitai Camera 

r rn rn jr rn jr rn jr rn 

Chat Rooms 

rrn jrrnjrrnjrrnjrrn 

Engineering Specific Software 

^ n r rn r rn c rn rn 

Generai Purpose Software (e.g., Office appiications) 

r rn r rn r rn r rn r r 

45 . How often do you carry the following technologies with you? (Mark one for each Item.) 

Never Sometimes Often Very frequently N/A 


Laptop computer 


r r'n r 

rn 

r 

rn 

jr 

rn .'r 

rn 

Cell Phone without Text Messaging 





r Cn r 

□ 

r 

r n 

jr 

rn jr 

r n 

Cell Phone with Text Messaging 






r Ca r 

r n 

r 

r n 

r 

rn T 

r n 

Personal Digital Assistant (PDA) (e.g. 

, Palm, Blackberry) 


r rn r 

□ 

r 

r □ 

r 

rn r 

r 

Scientific or Graphing Calculator 






r -'□ 

rn 

r 

rn 

r 

rn rr 

r n 


Generai-purpose Caicuiator 

r f'n r rn (' rn r rn jr rn 

Portabie Music Piayers (e.g., CD or MP3 piayer) 

r rn r rn r rn r rn jr rn 

Audio Recording Device 

r r'n , r rn _r rn r rn jr rn 

Digitai Camera 

r m -;r rn jr rn jr rn . r rn 


46 . List all of the electronic and computer technologies (i.e., like those listed in the previous question) you made even 
momentary use of yesterday and say what each was used for 

47 . Yesterday was a: (Mark one.) 

^ DWeek day 
^ nWeekend day 

48 . What is the first word or phrase you think of to describe your favorite professor? (Please respond in the space provided.) 

49 . Do you believe your peers would agree with this description: (Mark one) 

^ nYes 

DNo 

^ <"ni don't know 

50 . Some students participate In design competitions, Internships, and clubs. In the space provided identify engineering related 
activities you have participated In outside of class. (If this Is not applicable to you, please write “None.”) 

51 . What percentage of your friends in college (on this campus or other campuses) are studying engineering? (Mark one.) 

nNone 

^ nVery few 
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'■ ^ nLess than half 
nAbout half 
' (" DMore than half 

DAII or nearly all 


52 . Since entering this coiiege, indicate how often you have done the foiiowing with other engineering students: (Mark one for 
each item.) 

Not at all Rarely Occasionally Frequently 

Worked on class projects 

^ ^ 

Held a study group 

r c'n r -r rn jr To 

Took a specific lecture-type class 

r Tq r rn r Tq jr Tq 

Took a specific laboratory class 

r To r f'n .r rn jr rn 

Worked on homework 

("□ o rn jc Tq jr rn 

Reviewed class material 

<" □ ^ D Jf' rn jr rn 

Prepared for class exams 

r jr m jr Tq jr rn 

Wrote class reports 

r rn .jr rn jr rn jr rn 


53 . in what ways do you interact with other engineering students outside of the ciassroom? (if this is not applicable to you, 
please write “none.”) 

54 . In the space provided identify any classes (in high school or college so far) that have STRONGLY REINFORCED your 
interest in studying engineering. (If this is not applicable to you, write “none.”) 

55 . In the space provided identify any classes (in high school or college so far) that have STRONGLY WEAKENED your interest 
in studying engineering. (If this is not applicable to you, write “none.”) 

56 . What intellectual, personal, financial, and other challenges do you feel you may need to overcome to graduate with an 
engineering degree? If this is not applicable to you, write “none.” 

57 . In the space provided, list 5 terms you would use to describe “engineering”: 

58 . If you were asked the same question next week, how likely is it that you would list the same 5 terms? 

<^nNot likely 
^ DSomewhat likely 

^ DLlkely 

^ ^ nVery likely 

<" DExtremely likely 

59 . In the space provided, list 5 terms you would use to describe “design”: 

60 . If you were asked the same question next week, how likely is it that you would list the same 5 terms? 

(''□Not likely 
f' nSomewhat likely 

T' TnLikely 
‘ (" DVery likely 

DExtremely likely 
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Survey Design 


Help 

General Questions Order Preview Finish 

This is a preview of howtNs survey will look. In the preview the survey navigation buttons are inactive, use the section number buttons to 
view different sections. Some navigation buttons may not appear on your final survey, depending on what access it is assigned. The survey 
will use the background color of the document in which it is embedded. If you have no further changes click Finish at the bottom of this 

page. 


Academic Pathways Study Sprihs'05 Survey 

PLEASE CLICK THE SUBMIT BUTTCW CKLY AFTER YOU HAVE COMPLETED THE SURVEY. FOR BEST VIEWING 
RESULTS, PLEASE MAXIMIZE YOUR BROWSER WINDOW. 


1 of 16 


7/1/2005 11:12 AM 


4A-14 


APS Research Processes and Procedures 

March 2009 


phpESP.v 1.6.1 


hltp://caec-aps stanford.cdu/phpESP/admin/managc php 


INSTRUMENT STILL IN DEVELOPMENT - DO NOT DISTRIBUTE OUTSIDE APS! 

1 . What Is your expected year of graduation from college? 


O 2003 
O 2004 
O 2005 

O 2006 

O 2007 

O 2008 
O 2009 
O 2010 or later 


Z. Do you intend to complete a major in engineering? 

O Definitely Not 
^ Probably Not 
Q Not Sure 
0 Probably Yes 
Q Definitely Yes 


3, What do you intend to major In? 

Q Aeronautical Engineering 
0 Chemical Engineering 
Q Civil and Environmental Engineering 
Q Computer Science 
Q Electrical Engineering 
Q Management Science and Engineering 
Q Materials Science and Engineering 
Q Mechanical Engineering 
Q Other Engineering j 
0 Arts and Humanities 
0 Education 
0 Natural Science/Math 
0 Social Science 
0 Other Non-engineering 


4. If you intend to DOUBLE MAJOR, what Is the second major you intend to complete? (Mark N/A H you do not 
intend to double major.) 

0 Aeronautical Engineering 
0 Chemical Engineering 
0 Civil and Environmental Engineering 
0 Computer Science 
0 Electrical Engineering 
0 Management Science and Engineering 
0 Materials Science and Engineering 
0 Mechanical Engineering 
0 other Engineering j 
0 Arts and Humanities 
0 Education 
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Q Natural Science/Math 
O Social Science 
Q Other Non-engineering 

O N/A 


5. Do you intend to practice, conduct research In, or teach engineering for at least 3 years after graduation? 

Q Definitely Not 
Q Probably Not 
O Not Sure 
Q Probably Yes 
O Definitely Yes 


6. Xf you are thinking of going to graduate school NOT IN ENGINEERING, please mark your most probable area of 
study. Otherwise, mark H/A. 

Q Business 
Q Education 
0 Medicine 

O Law 

O MA/Ph.D. 

Q Public Service 
O Other 

O 


7. We are interested In knowing why you are studying engineering now. Please indicate below the extent to 
which the following reasons appiy to you: 


Not a Minimal Moderate Ma)or 
Reason Reason Reason Reason 


Technology plays an Important role in solving society's problems Q 
Engineers make more money than most other professionals 0 

My parent(s) would disapprove if I chose a ma)or other than ^ 

engineering ^ 

Engineers have contributed greatly to fixing problems in the world 0 
Engineers are well paid 0 
Engineering is an occupation that is respected by other people 0 
My parent(s) want me to be an engineer 0 
An engineering degree will guarantee me a ]ob when I graduate 0 
Engineers are creative problem solvers 0 
A faculty member, academic advisor, teaching assistant or other 
university affiliated person has encouraged and/or inspired me to 0 

Study engineering 

A non-university affiliated mentor has encouraged and/or inspired me ^ 

to study engineering LJ 


o o 
o o 
o o 

o o 
o o 
o o 
o o 
o o 
o o 

o o 
o o 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


8. Please indicate how strongly you disagree or agree with each of the statement: 


Disagree 

Strongly 


Disagree Agree 


I prefer studying in a group to studying by myself 0 


o o 


Agree 

Strongly 

o 


3 one 
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I am a competitive person 

o 

o o 

o 


I prefer working as part of a team to working alone 

o 

o o 

o 


I get along well with others in study situations 

o 

o o 

o 


I strive to get higher grades than my classmates 

o 

o o 

o 


The educational institution I am attending promotes competitive work 

o 

o o 

o 


My instructors often remind students that they need to do better than 
other students to obtain high grades 

o 

o o 

o 


I have easy access to work spaces where I can participate in peer 
study/discussion sessions with my fellow students 

o 

o o 

o 


I am encouraged by my instructors to initiate or participate in peer 
study sessions with my fellow students 

o 

o o 

o 


I am a collaborative person 

o 

o o 

o 


The educational institution I am attending promotes collaborative work 

o 

o o 

o 

d. 

Please indicate how strongly you disagree or agree with each of the statement: 




Disagree 

Strongly 

Disagree 

Agree 

^ strongly 


Creative thinking is one of my strengths Q 

o 

o o 



I am familiar with what a practicing engineer does Q 

o 

o o 



I am skilled at solving problems that can have multiple solutions Q 

o 

o o 


10. 

Rate yourself on each of the following traits as compared to your classmates. We want the most accurate 
estimate of how you see yourself. (Hark one In each row.) 


Lowest 

10% 

Averaoe 

Average Average 

Highest 

10% 


Self confidence (social) Q 

o 

o o 

o 


Leadership ability Q 

o 

o o 

o 


Public speaking ability Q 

o 

o o 

o 


Math ability Q 

o 

o o 

o 


Science ability Q 

o 

o o 

o 


Computer skills Q 

o 

o o 

o 


Communication skills Q 

o 

o o 

o 


Ability to apply math and science principles in solving real ^ 
world problems ^ 

o 

o o 

o 


Business ability Q 

o 

o o 

o 


Ability to perform in teams Q 

o 

O 0 

o 


Critical Thinking skills Q 

o 

o o 

o 

11. 

How important do you think each of the following skills and abilities is to becoming a successful engineer? 
(Mark one In each row.) 


Not somewhat very crucial 

Important Important Important 


Self confidence (social) Q 

o 

o 

o 


Leadership ability Q 

o 

o 

o 


Public speaking ability Q 

o 

o 

o 


Math ability Q 

0 

0 

0 


Science ability Q 

o 

o 

o 


Computer skills ^ 

0 

0 

0 


Communication skills Q 

o 

0 

o 
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Ability to apply math and science principles In solving real 
world problems 
Susiness ability 
Ability to perform in teams 


o 

o 

o 


o 

o 

o 


o o 

o o 

o o 


12. Please rate your satisfaction with this Institution on each of the aspects of campus life listed below. Xf you do 
not have experience with this aspect, mark N/A. 



Very 

Dissatisfied 

Dissatisfied 

Satisfied 

Very 
Sa tisfied 

N/A 

Quality of instruction by faculty 

o 

o 

0 

o 

o 

Quality of advising by faculty 

o 

o 

o 

o 

o 

Availability of faculty 

o 

o 

o 

o 

o 

Quality of instruction by teaching assistants 

o 

o 

o 

o 

o 

Quality of advising by teaching assistants 

o 

o 

o 

o 

o 

Availability of teaching assistants 

o 

o 

o 

o 

o 


13. Please rate your satisfaction with each of the following at this institution. If you do not use the service or 
facility, mark N/A. 



very 

Dissatisfied 

Dissatisfied 

satlsritd 

very 

Setlsried 

N/A 

Computer facilities 

o 

o 

O 

O 

O 

Libraries 

o 

0 

O 

O 

o 

Classrooms 

o 

o 

O 

o 

o 

Tutoring 

o 

o 

O 

o 

o 

Academic advising 

o 

0 

O 

O 

o 

Laboratories 

o 

o 

O 

O 

o 


14. Since January, how often have you taken courses which required your engagement in individual and/or group 
projects? 

Q Never 
O Rarely 
Q Occasionally 
Q Frequently 


15. Think about the engineering classes you have taken since January (engineering, math, and science classes). 
Indicate how often you: (Mark N/A if you have not taken any engineering related classes.) 


Never 

Came late to engineering class ^ 

Skipped engineering class ^ 

Turned in engineering assignments that did not ^ 
reflect your best work 

Turned In engineering assignments late Q 

Thought engineering classes were boring Q 


Rarely Occasionally Frequently N/A 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 


Softs 
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16. Think about the liberal arts classes you have taken since January (not engineering, math, and science). 
Indicate how often you: (Hark N/A if you have not taken any non>englneerlng related classes.) 


Never 

Came late to liberal arts class Q 
Skipped liberal arts class Q 

Turned In liberal arts assignments that did not 
reflect your best vMrk 

Turned In liberal arts assignments late Q 
Thought liberal arts classes were boring Q 


Rarely Occasionally Frequently N/A 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 


17. How often have you interacted with the following people since January(e.g. by phone, e-mail. Instant 
Messenger, or in person)? (Mark one for each item.) 



Never 

1-2 

times 

1-2 

times 

Once 

per 

2-3 

times 

Dally 



per 

per 

Week 

per 



Term 

Month 

week 


Faculty during class 

o 

o 

o 

o 

O 

o 

Faculty during office hours 

o 

o 

o 

o 

0 

o 

Faculty outside of class or office hours 

o 

o 

o 

o 

O 

o 

Teaching Assistants during class 

o 

o 

o 

o 

0 

o 

Teaching Assistants during office hours 

o 

o 

o 

o 

O 

o 

Teaching Assistants outside of class or office hours 

o 

o 

o 

o 

0 

o 


18. What portion of the courses you have taken since January have been taught primarily by graduate students? 

Q None 
0 very little 
O Less than half 
0 About half 
0 More than half 
0 All or nearly all 


19. Since January, what portion of your classes used the following teaching methods? 



None 

very 

little 

Less 

than 

half 

About 

half 

More 

than 

half 

All or 
nearly 
all 

Lectures 

o 

o 

o 

o 

o 

o 

Individual Projects 

o 

o 

o 

o 

o 

o 

Team Projects 

o 

o 

o 

o 

o 

o 

Labs 

o 

o 

o 

o 

o 

o 

Seminars 

o 

o 

o 

o 

o 

o 


20. To what extent have your courses required your engagement in Individual and/or group projects? 


0 Too Few 
0 Enough 
0 Too many 
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21. Some people are involved in non-engineering activities on or off campus, such as hobbies, civic or church 
organizations, campus publications, student government, social fraternity or sorority, sports, etc. How 
important Is it for you to be involved in these kind of activities? 


Q Not Important 
O Somewhat Important 
Q Very Important 
0 Essential 


22. How often are you involved in the kinds of non-engIneerIng activities described above? 

O Never 
0 Rarely 
0 Occasionally 
0 Frequently 


23. Thinking about your college experience since January, please indicate how much pressure you are feeling 
related to the following: 


No Reasonable Extreme 
Pressure Pressure Pressure 


Course load (amount of course material being covered) 0 

Course pace (the pace at which the course material is being covered) 0 

Balance between social and academic life 0 


o o 

o o 

o o 


24. How well are you meeting the workload demands of your coursework? 

0 I am meeting all of the demands easily 
0 I am meeting all of the demands, but it is hard work 
0 I am meeting most of the demands, but cannot meet some 
0 I can meet some of the demands, but cannot meet most 
0 I cannot meet any of the demands 


25. How stressed do you feel in your coursework right now? 

0 No stress 
Some stress 
0 Reasonable stress 
0 significant stress 
0 Extreme stress 


26. Do you have any concern about your ability to finance your college education? 

(^i None (I am confident that I will have sufficient funds) 

0 Some (but I probably will have sufficient funds) 

0 Major (not sure if I will have sufficient funds to complete college) 
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27. How do you meet your college expenses? 




Very 

little 

Less 

About 

half 

More 

All or 


None 

than 

half 

than 

half 

nearly 

all 

Self (income) 

o 

o 

o 

o 

o 

o 

Self (savings) 

o 

o 

o 

o 

o 

o 

Parents and family 

o 

o 

o 

o 

o 

o 

Employer support 

o 

o 

o 

o 

o 

o 

Schalorships and grants 

o 

o 

0 

o 

o 

o 

Loans 

o 

o 

0 

o 

o 

o 


26, Do you have family members who are practicing engineers? 

O 

o »<■ 


29. Do you have close friends who are practicing engineers? 


O Yes 
O 


30. How much exposure have you had to a professional engineering environment as a visitor, Intern, or 
employee? 

O No exposure 
0 Limited exposure 
0 Moderate exposure 
0 Extensive exposure 


31. About how many hours do you spend In a typical 7-day week doing each of the following? 


Preparing for class (studying, reading, writing, 
doing homework or lab work, analyzing data, 
rehearsing, and other academic activities) 
Working for pay 
Participating in co-curricular activities 
(organizations, campus publications, student 
government, social fraternity or sorority, 
intercollegiate or intramural sports, etc.) 
Relaxing and socializing (watching TV, partying, 
exercising, etc.) 
Providing care for dependents living with you 
(parents, children, spouse, etc.) 
Commuting to class (driving, walking, etc.) 


more 


0 

1-5 

6-10 

11-15 

16-20 

21-25 

26-30 

than 

30 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


32. Please rate the overall quality of your collegiate experience so far: 
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O Very dissatisfied 
O Dissatisfied 
0 Satisfied 
O Very satisfied 


33. Your sex: 

0 Male 
0 Female 


34. Please Indicate your ethnic background: (Mark all that apply) 

O White/Caucasian 
G African American/Black 
G American indian/Alaska Native 
G Asian American/Asian 
G Native Hawaiian/Pacific islander 
G Mexican American/Chicano 
G Puerto Rican 
G Other Latino 
G Other 


35. Citizenship Status: 

0 U.S. Resident 

0 Permenant Resident (Green Card) 
0 Neither 


36. What was your average grade In high school? (Mark one) 

0 A or A-f 

OA- 

o 

O B 
O B- 

Oc* 

O c 
O o 


37. What is the highest level education that your mother completed? (Hark one) 

0 Did not finish high school 
0 Graduate from high school 
0 Attended college but did not complete degree 
0 Completed an Asssociate's degree (A. A., A.S., etc.) 

0 Completed a Bachelor's degree (B.A., B.S., etc.) 


9 one 
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Q Completed a Master's degree (M.A., M.S. 

etc.) 







Q Completed a Professional degree (3.0., M.O., etc.) 







Q Complete a Doctoral degree (Ph.O.) 







38. 

Please rate the extent to which you agree that each of the following Is reason that you are currently majoring 
in or considering majoring In engineering: 



Strongly Moderately 
Disagree Disagree 

Disagree 

unsure 

Agree 

Moderately 

Agree 

strongly 

Agree 


I think engineering is interesting 

o 

o 

o 

o 

o 

o 

o 


I am majoring in engineering for 
my own good 

0 

o 

0 

0 

o 

0 

0 


I am supposed to major in 
engineering 

o 

o 

o 

o 

o 

o 

o 


There may be good reasons to 
major in engineering, but 
personally, I don't see any 

o 

o 

o 

o 

o 

o 

o 


I think engineering is pleasant 

o 

o 

o 

o 

o 

o 

o 


I think engineering is good for me 

o 

o 

o 

o 

o 

o 

o 


Majoring In engineering is 
something that I have to do 

o 

o 

o 

o 

o 

o 

o 


I am majoring In (considering 
majoring in) engineering, but I 
am not sure If It Is worth it 

o 

o 

o 

o 

o 

o 

o 


Majoring in engineering is fun 

o 

o 

o 

o 

o 

o 

o 


It is my personal decision 

o 

o 

o 

o 

o 

o 

o 


I don't have any choice 

o 

o 

o 

o 

o 

o 

o 


I don't know. I don't see what the 
activity brings me 

o 

o 

o 

o 

o 

o 

o 


I feel good when I am doing 
engineering activities 

o 

o 

o 

o 

o 

o 

o 


I believe engineering is important 
for me 

o 

o 

o 

o 

o 

o 

o 


I feel that I have to do It 

o 

o 

o 

o 

o 

o 

o 


I am doing It, but am not sure it 
is a good thing to pursue 

o 

o 

o 

o 

o 

o 

o 

39. 

Please rate the extent to which you agree with the following statements: 






strongly Moderately 
disagree Disagree 

Disagree 

unsure 

Agree 

Moderately 

Agree 

strongly 

Agree 


On the whole, I am satisfied with 
myself 

o 

o 

o 

o 

o 

o 

o 


At times, I think I am no good at 
all 

o 

o 

o 

o 

o 

o 

o 


I feel that I have a number of 
good qualities 

o 

o 

o 

o 

o 

o 

o 


I am able to do things as well as 
most other people 

o 

o 

o 

o 

o 

o 

o 


I feel I do not have much to be 
proud of 

o 

o 

o 

o 

o 

o 

o 


I certainly feel useless at times 

o 

o 

o 

o 

o 

o 

o 


I feel that I'm a person of worth, 
at least on an equal plane with 
others 

o 

0 

0 

0 

0 

0 

0 


I wish I could have more respect 
for myself 

o 

o 

o 

o 

o 

o 

o 


All In all, I am Inclined to feel that 
lama failure 

o 

o 

o 

o 

o 

o 

o 
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I Uke a positive attitude toward 
myself 

o 

o 

o 

o 

o 

o 

o 

40. Please rate the extent to which you agree with the following statements. 





Strong ly 
Disagree 

Moderately 

Disagree 

Disagree 

Unsure 

Agree 

Moderately 

Agree 

Strongly 

Agree 

Overall, being a member of my 
ethnic group has very little to do 
with how I feel about myself 

o 

o 

o 

o 

o 

o 

o 

In general, being a member of 
my ethnic group is an important 
part of my self-image 

o 

o 

o 

o 

o 

o 

o 

My destiny is tied to the destiny 
of other members of my ethnic 
group 

o 

o 

o 

o 

o 

o 

o 

Being a member of my ethnic 
group is unimportant to my sense 
of what kind of person I am 

o 

o 

o 

o 

o 

o 

o 

I have a strong sense of 
belonging to my ethnic group 
community 

o 

o 

o 

o 

o 

o 

o 

I have a strong attachment to 
other members of my ethnic 
group 

o 

o 

o 

o 

o 

o 

o 

Being a member of my ethnic 
group is an important reflection 
of who I am 

o 

o 

o 

o 

o 

o 

o 

Being a member of my ethnic 
group IS not a major factor in my 
social relationships 

o 

o 

o 

o 

o 

o 

o 

Overall, being a member of my 
gender has very little to do with 
how I feel about myself 

o 

o 

o 

o 

o 

o 

o 

In general, being a member of 
my gender is an important part of 
my self-image 

o 

o 

o 

o 

o 

o 

o 

My destiny is tied to the destiny 
of other members of my gender 

o 

o 

o 

o 

o 

o 

o 

Being a member of my gender is 
unimportant to my sense of what 
kind of person I am 

o 

o 

o 

o 

o 

o 

o 

I have a strong sense of 
belonging to my gender 
community 

o 

o 

o 

o 

o 

o 

o 

I have a strong attachment to 
other members of my gender 

o 

o 

o 

o 

o 

o 

o 

Being a member of my gender is 
an important reflection of who I 
am 

o 

o 

o 

o 

o 

o 

o 

Being a member of my gender is 
not a major factor In my social 
relationships 

0 

o 

o 

o 

o 

o 

o 

41. Please rate the extent to which you agree with the following statements: 





strongly Moderately 
disagree Disagree 

Disagree 

Unsure 

Agree 

Moderately 

Agree 

strongly 

Agree 

Overall, being an engineering 
student has very little to do with 
how I feel about myself 

o 

o 

o 

o 

o 

o 

o 
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In general, being an engineering 


student is an important part of my 
self-image 

o 

o 

o 

o 

o 

o 

o 

My destiny is tied to the destiny of 
other engineering students 

o 

o 

o 

o 

o 

o 

o 

Being an engineering student is 
unimportant to my sense of what 
kind of person lam 

o 

o 

o 

o 

o 

o 

o 

I have a strong sense of 
belonging to the engineering 
student community 

o 

o 

o 

o 

o 

o 

o 

I have a strong attachment to 
other engineering students 

o 

o 

o 

o 

o 

o 

o 

Being an engineering student Is 
an important reflection of who I 
am 

o 

o 

o 

o 

o 

o 

o 

Being an engineering student is 
not a major factor in my social 
relationships 

o 

o 

o 

o 

o 

o 

o 

I feel good about engineers 

o 

o 

o 

o 

o 

o 

o 

I am happy that I am going to be 
an engineer 

o 

o 

o 

o 

o 

o 

o 

I feel that engineers have made 
major accomplishments and 
advancements 

o 

o 

0 

0 

o 

o 

o 

I often regret that lam going to 
become an engineer 

o 

o 

o 

o 

o 

o 

o 

lam proud to be an engineer 

o 

o 

o 

o 

o 

o 

o 

I feel that the engineering 
community has made valuable 
contributions to this society 

o 

o 

o 

o 

o 

o 

o 

Overall, engineers are considered 
good by others 

o 

o 

0 

o 

o 

o 

o 

In general, others respect 
engineers 

o 

o 

o 

o 

o 

o 

o 

Most people consider engineers, 
on the average, to be more 
ineffective than other 
professionals 

o 

o 

o 

o 

o 

o 

o 

Engineers are not respected by 
the broader society 

o 

o 

o 

o 

o 

o 

o 

In general, other professionals 
view engineers in a positive 
manner 

o 

o 

o 

o 

o 

o 

o 

Society views engineers as an 
asset 

o 

o 

o 

o 

o 

o 

o 

I identify with engineering 
students 

o 

o 

o 

o 

o 

o 

o 

I am glad to belong to a group of 
engineering students 

o 

o 

o 

o 

o 

o 

o 

I feel held back by engineering 
students 

o 

o 

o 

o 

o 

o 

o 

I think engineering students work 
well together 

o 

o 

o 

o 

o 

o 

o 

I see myself as an important part 
of engineering students on 
campus 

o 

o 

o 

o 

o 

o 

o 

I fit in well with the other 
engineering students 

o 

o 

o 

o 

o 

o 

o 

I consider engineering students to 
not be important 

o 

o 

o 

o 

o 

o 

o 

I feel uneasy with other 
engineering students 

o 

o 

o 

o 

o 

o 

o 

I feel strong ties to engineering 
students 

o 

o 

o 

o 

o 

o 

o 
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42. Of the twenty-three design activities below, please put a check mark next to the she MOST IMPORTANT. 

^ Abstracting 
D Brainstorming 
O Building 
O Communicating 
l~1 Decomposing 
O Evaluating 
G Generating atternatives 
□ Goal setting 
O Identifying Constraints 
n Imagining 
O Iterating 
G Making decisions 
G Making trade-offs 
G Modeling 
G Planning 
G Prototyping 
G Seeking Information 
G Sketching 
G Synthesizing 
G Testing 

G Understanding the problem 
G Using creativity 
G Visualizing 


43. Of the twenty-three design activities below, please put a check mark next to the sbr LEAST IMPORTANT. 

G Abstracting 
G Brainstorming 
G Building 
G Communicating 
G Decomposing 
G Evaluating 
G Generating alternatives 
G Goal setting 
G Identifying constraints 
G Imagining 
G Iterating 
G Making decisions 
G Making trade-offs 
G Modeling 
G Planning 
G Prototyping 
G Seeking Information 
G Sketching 
G Synthesizing 
G Testing 
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G Understanding the problem 
0 Using creativity 
G visualizing 


Zn questions 44>48« respond five times to the question "Who Am 17" ("Answer A" should be for your first response, 
"Answer B" for your second response, "Answer C" for your third response, "Answer D" for your fourth response, and 
“Answer E" for your fifth response.) 

44. 

Who am I? (Answer A) 


45. 

1 

Who am 17 (Answer B) 


46. 

1 

Who am I? (Answer C) 


47. 

1 

Who am 17 (Answer D) 


48. 

1 

Who am 1? (Answer E) 


49. 

1 

Rank order your answer to the previous question In terms of how important It is to the way you generally feel 
about yourself. For each of your answers, mark a rank between 1 and 5. A rank of "I" Indicates the most 
important component, and "5" Indicates the least Important component. 


12 3 

4 5 


Answer A Q Q Q 

0 o 


Answer B Q O O 

o o 


Answer C O O O 

o o 


Answer D O O O 

o o 


Answer E Q O O 

o o 

50. 

For the following engineering design activities, please indicate your level of confidence. For example, if you 
have little or no confidence In your ability to model engineering solutions, then mark poor. If you are 
extremely confident in your ability, mark excellent. 



Poor Fair Good Excellent 



Defining what the problem really Is O O O O O 
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Searching for and collecting Information needed to solve the 

problem 

o 

o 

o 

o 

o 


Thinking up potential solutions to the problem 

0 

o 

0 

0 

0 


Detailing how to build the solution to the problem 

0 

o 

o 

0 

o 


Assessing and passing judgment on a possible or planned solution 

to the problem 

0 

o 

o 

o 

o 


Comparing and contrasting two solutions to the problem on a 
particular dimension such as cost 

o 

o 

o 

o 

o 


Selecting one Idea or solution to the problem from among those 

considered 

o 

o 

o 

o 

o 


Communicating elements of the design In sketches, diagrams, lists, 
and written or oral reports 

o 

o 

o 

o 

o 

51. 

For the following engineering design activities, please Indicate how often you engaged In the activity In your 
coursework In the current academic year. 


Never 

1-2 

times 

per 

term 

1-2 

times 

a 

month 

Once 

a 

week 

2-3 

times 

a 

week 

Dally 


Defining what the problem really is Q 

o 

o 

o 

o 

o 


Searching for and collecting information needed to solve the ^ 

problem ^ 

o 

o 

o 

o 

o 


Thinking up potential solutions to the problem Q 

o 

o 

o 

o 

o 


Detailing how to build the solution to the problem ^ 

o 

o 

o 

o 

o 


Assessing and passing judgment on a possible or planned p. 

solution to the problem 

o 

o 

o 

o 

o 


Comparing and contrasting two solutions to the problem on a ^ 
particular dimension such as cost ^ 

o 

o 

o 

0 

o 


Selecting one Idea or solution to the problem from among ^ 
those considered ^ 

o 

o 

o 

o 

o 


Communicating elements of the design in sketches, diagrams, p. 

lists, and written or oral reports ^ 

o 

o 

o 

o 

o 

52. 

For the following engineering design activities, please Indicate how well you think your courses are preparing 
you to engage In the activity. For example. If you think they are not preparing you at all, then mark poor. If 
you think they are preparing you extremely well, then mark excellent. 


Poor 

Fair 

Wall 

Excellent 

well 


Defining what the problem really is 

o 

o 

o 

o 

o 


Searching for and collecting Information needed to solve the 

problem 

o 

o 

o 

o 

o 


Thinking up potential solutions to the problem 

0 

o 

0 

0 

0 


Detailing how to build the solution to the problem 

o 

o 

o 

o 

o 


Assessing and passing judgment on a possible or planned solution 

to the problem 

o 

o 

o 

o 

o 


Comparing and contrasting two solutions to the problem on a 
particular dimension such as cost 

o 

o 

o 

o 

o 


Selecting one Idea or solution to the problem from among those 

considered 

o 

o 

o 

o 

o 


Communicating elements of the design in sketches, diagrams, lists, 
and written or oral reports 

o 

o 

o 

o 

o 

53. 

What are your summer plans? 
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The survey title end other general fields are on the General tab. Individual survey questions are added and modified on the Questions tab. 
Questions may be re-ordered or deleted from the Order tab. You may see a preview of your survey at any time, by going to the Preview tab. If 
you have no further changes click Finish to go back to the Management interface. 

Click here to open the Help windov/. 

^GenwalJ ^OuestaosJ ^Orde^ ^PrewwJ ^ini^ij 
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Survey Design 

Help 

General Questions Order Preview ;Finish 

This is a preview of how this survey will look, In the preview the survey navigation buttons are inactive, use the section number buttons to 
view different sections, Some navigation buttons may not appear on your final survey, depending on what access it is assigned. The survey 
will use the background color of the document in which it is embedded. If you have no further changes click Finish at the bottom of this 

page. 


Academic Pathways Study Sprimg'07 Survey 

Stamford Uhiversity 

Please click the [Submit Survey] button to access the next page of the survey, and oniy 
after you have completed this page. 

For best viewing results, please maximize your browser window. 


1, What is your expected year of graduation from college? 

O 2007 

O 2008 

O 2009 
O 2010 or later 


2, Do you intend to complete a major in engineering? 

O Definitely Not 
Q Probably Not 
O Not Sure 
O Probably Yes 
Q Definitely Yes 


3, What doyou intend to major in? 

Q Aeronautical Engineering 
Q Chemical Engineering 
O Civil and Environmental Engineering 
O Computer Science 
Q Electrical Engineering 
Q Management Science and Engineering 
O Materials Science and Engineering 
O Mechanical Engineering 
O Other Engineering | 

Q Arts and Humanities 
Q Education 
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O Natural Science/Math 
O Social Science 
O Other Non-engineering f 


4. If you Intend to DOUBLE MAJOR, what Is the second major you Intend to complete? (Mark N/A If you do not 
Intend to double major.) 

^ Aeronautical Engineering 
O Chemical Engineering 
Q Civil and Environmental Engineering 
O Computer Science 
O Electrical Engineering 
O Management Science and Engineering 
O Materials Science and Engineering 
O Mechanical Engineering 
O Other Engineering | 

O Arts and Humanities 
O Education 
O Natural Science/Math 
O Social Science 
O Other Non-engineering | 

O N/A 


5. Do you Intend to practice, conduct research In, or teach engineering for at least 3 years after graduation? 

O Definitely Not 
O Probably Not 
O Not Sure 
O Probably Yes 
O Definitely Yes 


6. Iff you are thinking of going to graduate school in a field OTHER THAN engineering, please mark your most 
probable area of study, otherwise, mark N/A. 

O Business 
O Education 

Q Humanities and Social Sciences 

O Law 

O Medicine 

^ Natural Sciences/Math 
O Public Service 
^ Other 

O N/A 


7. We are interested In knowing why you are studying engineering now. Please Indicate below the extent to 
which the following reasons apply to you: 
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Not a Minimal Moderato 
Reason Reason Reason 

Major 

Reason 


Technology plays an Important role In solving society's problems 

o o 

o 

o 


Engineers make more money than most other professionals 

o o 

o 

o 


My parent($) would disapprove If I chose a major other than 

engineering 

o o 

o 

0 


Engineers have contributed greatly to fixing problems in the world 

o o 

o 

o 


Engineers are well paid 

o o 

o 

0 


My parent(s) want me to be an engineer 

o o 

o 

o 


An engineering degree will guarantee me a Job when I graduate 

o o 

o 

0 


A faculty member, academic advisor, teaching assistant or other 
university affiliated person has encouraged and/or Inspired me to 

study engineering 

o o 

o 

o 


A non-university affiliated mentor has encouraged and/or inspired me 

to study engineering 

o o 

o 

o 

8. 

Please Indicate how strongly you disagree or agree wtth each of the statements: 




Disagree 

Strongly Disagree Agree 

Agree 

Strongly 


Creative thinking Is one of my strengths Q 

o o 

o 



1 am familiar with what a practicing engineer does O 

o o 

o 



I am skilled at solving problems that can have multiple solutions Q 

o o 

o 


9. 

Rate yourself on each of the following traits as compared to your classmates. We want the most accurate 
estimate of how you see yourself. (Mark one In each row.) 


Lowest 

10% 

Below Above 

Average Average Average 

Highest 

10% 


Self confidence (social) O 

o o 

o 

o 


Leadership ability O 

o o 

o 

o 


Public speaking ability O 

o o 

o 

o 


Math ability O 

o o 

o 

o 


Science ability 0 

o o 

o 

o 


Communication skills O 

o o 

o 

o 


Ability to apply math and science principles In solving real world 

problems O 

o o 

o 

o 


Business ability O 

o o 

o 

o 


Ability to perform In teams Q 

o o 

o 

o 


Critical thinking skills Q 

o o 

o 

o 

10. 

How important do you think each of the following skills and abilities is to becoming a successful engineer? 
(Mark one in each row.) 


Not Somewhat Very 
Important Important Important Crucial 


Self confidence (social) Q 

o 

o 

o 


Leadership ability Q 

o 

o 

o 


Public speaking ability Q 

o 

o 

o 


Math ability Q 

o 

o 

o 


Science ability Q 

o 

o 

o 
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Communtcation skills 

o 

o 

o 

o 


Ability to apply math and science principles in solving real 
world problems 

o 

o 

o 

o 


Business ability 

o 

o 

o 

o 


Ability to perform In teams 

o 

o 

o 

o 

11. 

Please rate your satisfaction with this institution on each of the aspects of campus life listed below. If you do 
not have experience with this aspect, mark N/A. 


Very Very 

Dissatisfied Dissatisfied Satisfied Satisfied 

N/A 



Quality of Instruction by faculty O O 

o 

o 

o 



Quality of advising by faculty O O 

o 

o 

o 



Availability of faculty O O 

o 

o 

o 



Quality of Instruction by teaching assistants O O 

o 

o 

o 



Quality of advising by teaching assistants O O 

o 

o 

o 



Availability of teaching assistants O O 

o 

o 

o 


12. 

Please rate your satisfaction with each of the following at this institution. If you do not 
N/A. 

use this facility, mark 


Very Very 

Dissatisfied Dissatisfied Satisfied Satisfied 

N/A 





Computer facilities O O O O 

o 





Libraries O O O O 

o 





Classrooms O O O O 

o 





Laboratories O O O O 

o 




13. 

Think about the engineering classes you have taken during the current school year (engineering, math, and 
science classes). Indicate how often you: (Mark N/A If you have not taken any engineering related classes.) 


Never 

Rarely 

Occasionally Frequently 

N/A 


Came late to engineering class O 

o 

o 

o 

o 


Skipped engineering class O 

o 

o 

o 

o 


Turned in engineering assignments that did not 

reflect your best work Q 

o 

o 

o 

o 


Turned In engineering assignments late ^ 

0 

o 

o 

o 

14. 

Think about the liberal arts classes you have taken during the current school year (not engineering, math, and 
science). Indicate how often you: (Mark N/A if you have not taken any non*englneerlng related classes.) 


Never 

Rarely 

Occasionally Frequently 

N/A 


Came late to liberal arts class Q 

o 

o 

o 

o 


Skipped liberal arts class O 

o 

o 

o 

o 


Turned in liberal arts assignments that did not 

reflect your best work O 

o 

o 

o 

o 


Turned In liberal arts assignments late Q 

o 

o 

o 

o 
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15. How often have you interacted with the following people during the current school year (e.g. by phone, 
e^maH Instant Messenger, or In person)? (Mark one for each Item.) 




1-2 

1-2 


2-3 




times 

times 

Once 

times 




per 

per 

per 

per 



Never 

Term 

Month 

Week 

Week 

Dally 

Faculty during class 

o 

o 

o 

o 

o 

o 

Faculty during office hours 

o 

o 

o 

o 

o 

o 

Faculty outside of class or office hours 

o 

o 

o 

o 

o 

o 

Teaching Assistants during class 

o 

o 

o 

o 

o 

0 

Teaching Assistants during office hours 

o 

o 

o 

o 

o 

o 

Teaching Assistants outside of class or office hours 

0 

0 

o 

o 

o 

o 


16. What portion of the courses you have taken during the current school year have been taught primarily by 
graduate students? 

O None 
O Very little 

O l-sss 

(2 About half 
O than half 
O All or nearly all 


17. During the current school year, what portion of your classes have used the following teaching methods? 





Less 


More 

All or 



Very 

than 

About 

than 

nearly 


None 

littia 

half 

half 

half 

all 

Lectures 

o 

O 

o 

o 

o 

o 

Individual Projects 

o 

O 

o 

o 

o 

o 

Team Projects 

o 

O 

o 

o 

o 

o 

Labs 

o 

O 

o 

o 

o 

o 

Seminars 

o 

O 

o 

o 

o 

o 


18. Some people are involved in non-engineering activities on or off campus, such as hobbies, civic or church 
organizations, campus publications, student government, social fraternity or sorority, sports, etc. How 
important is it for you to be involved in these kind of activities? 

O Not Important 
Q Somewhat Important 
Q Very Important 
O Essential 


19. How often are you involved In the kinds of non-engineering activities described above? 


O Never 
O Rarely 
O Occasionally 
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O Frequently 


20. Thinking about your coilege experience during the current school year, please indicate how much pressure 
you are feeling related to the following: 


No Reasonable Extreme 
Pressure Pressure Pressure 


Course load (amount of course material being covered) O 

Course pace (the pace at which the course material ts being covered) O 

Balance between social and academic life O 


o o 

o o 

o o 


21. How well are you meeting the workload demands of your coursework? 

O I meeting all of the demands easily 
Clem meeting all of the demands, but It Is hard work 
0 I am meeting most of the demands, but cannot meet some 
C I can meet some of the demands, but cannot meet most 
0 I cannot meet any of the demands 


22. How stressed do you feel In your coursework right now? 

0 No stress 
0 Some stress 
0 Reasonable stress 
0 Significant stress 
0 Extreme stress 


23. DO you have any concerns about your ability to finance your college education? 

0 None (I am confident that I will have sufficient funds) 

0 Some (but I probably will have sufficient funds) 

0 Major (not sure If I will have sufficient funds to complete college) 


24. How do you meet your college expenses? 





Less 


More 

All or 



Very 

than 

About 

than 

nea rly 


None 

llltle 

half 

half 

half 

all 

Seif (income) 

o 

O 

o 

o 

o 

o 

Self (savings) 

o 

O 

o 

o 

o 

o 

Parents and family 

o 

O 

o 

o 

o 

o 

Employer support 

o 

o 

o 

o 

o 

o 

Scholarships and grants 

o 

o 

o 

o 

o 

o 

Loans 

o 

o 

o 

o 

o 

o 
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25. How much exposure have you had to a professional engineering environment as a visitor, intern, or 
employee? 

O No exposure 
0 Limited exposure 
0 Moderate exposure 
0 Extensive exposure 


26. About how many hours do you spend In a typical 7>day week doing each of the following? 


0 

Preparing for class (studying, reading, writing, 
doing homework or lab work, analyzing data, 
rehearsing, and other academic activities) 0 
Working for pay 0 
Participating in co-curricular activities 
(organizations, campus publications, student 
government, social fraternity or sorority, 
Intercollegiate or intramural sports, etc.) 0 
Relaxing and socializing (watching TV, partying, 

exercising, etc.) 0 

Providing care for dependents living with you 

(parents, children, spouse, etc.) 0 
Commuting to class (driving, walking, etc.) 0 


6-10 


11-15 16-20 21-25 26-30 


more 

than 

30 


o 

o 


o 

o 


o 

o 


o 

o 


o 

o 


o 

o 


o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


o 

o 

o 

o 


27. Please rate the overall quality of your collegiate experience so far: 

0 Very dissatisfied 
O Dissatisfied 
0 Satisfied 
0 Very saUsfled 


28. Which of the following statements best describes your situation with respect to an engineering major? (Mark 
one.) 


I started at this institution. .. 

0 Intending to major In engineering and never doubted the decision. 
0 Intending to major in engineering and have/had doubts. 

0 Considering engineering, but was open to other majors, too. 

0 Intending another major, but am now considering engineering. 

0 Completely undecided about what my major would be. 


29. My decision to CONTINUE with an engineering major prlm»rUy came from: (Nark one.) 


0 Not applicable - 1 am not continuing in an engineering major. 

Experiences with PRE-ENGXNEERXNG-related (math, physKs, etc) coursework, faculty, and/or 
0 research/lntern$hlp(s) 
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O Exp«riences with ENGlNEERlNG>related coursework, faculty, and/or res«arch/lnternship(s) 
O Experiences with OTHER coursework, faculty, and/or research/lnternshlp(s) 


30. Hy DOUBTS about continuing in an engineering major prtmarUy came from: (Hark one.) 

O Not applicable • I never doubted continuing In an engineering major. 

Experiences with PRE«ENGINEERING-related (math, physics, etc) coursework, faculty, and/or 
O research/lnternshlp(s) 

O Experiences with ENGINEERING-related coursework, faculty, and/or research/internship(s) 

O Experiences with OTHER coursework, faculty, and/or research/internship($) 


31. Since coming to college, have you had any research experlence(s)7 

o No 

O Yes, In engineering related areas 

O Yes, In non*englneerlng related areas 

O Yes, In both engineering and non-engineering related areas 


32. What is your expected grade point average this academic term? 

o Aor A+ (3.9-4.0) 

O A- (3. 5-3.8) 

C B+(3.2-3.4) 

O B (2.9-3. 1) 

O B- (2.5-2.S) 

O C+ (2. 2-2.4) 

O C (1.9-2.1) 

O C- (1. 5-1.8) 

O D (less than 1.4) 


33. Your sex: 

o Male 
O Female 


34. How old will you be on December 31st of this year? (Mark one.) 
Q 20 or younger 

O 21 
O 22 

C 23 
O 24 
O 25-29 

O 20-54 
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O 35-39 

O <0-5t 

O 55 or older 


35. Please Indicate your ethnic background: (Mark all that apply.) 

G white/caucasian 
G African Amerkan/Black 
G American Indian/Alaska Native 
G Asian American/Asian 
G Native Hawaiian/Pacific islander 
G Mexican American/Chicano 
G Puerto Rican 
G other Latino 
G Other 


36. What Is your marital status? 

O Not married 
0 Married 
O Divorced 
O Separated 
O Widowed 


37. How many dependents do you have? 
0 None 

O 1 

O 2 

O 3 

o« 

C 5 or more 


38. Are you: 

o A U.S. Citizen 

O A Permanent Resident of the U.S. 
O Other 


39. Were you born In the United states? 
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O Y«$ 

O If at what age did you immigrate to the U.S.? f 


40. Did oit« or more of your parenU/guardians immigrate to the United States? 

O Yes 

O No 


41. Is English your first language? 
O Yes 

O NO 


42. What is the highest level of education that your mother completed? (Mark one.) 

O Did not finish high school 
O Graduated from high school 
O Attended college but did not complete degree 
O Completed an Associate's degree (AA, AS, etc.) 

C Completed a Bachelor's degree (BA, BS, etc.) 

O Completed a Master's degree (MA, MS, etc.) 

O Completed a Doctoral or Professional degree (JD, MD, PhD, etc.) 


43. What is the highest level of education that your father completed? (Hark one.) 

O Did not finish high school 
0 Graduated from high school 
0 Attended college but did not complete degree 
0 Completed an Associate's degree (AA, AS, etc.) 

0 Completed a Bachelor's degree (BA, 8S, etc.) 

C Completed a Master's degree (MA, MS, etc.) 

0 Completed a Doctoral or Professional degree (3D, MD, PhD, etc.) 


44. What is your best estimate of your parents' total Income last year? consider Income from all sources before 
taxes. (Mark one.) 

0 Less than $10,000 
0 $10,000-$14,999 
0 $15,000-$19,999 
0 $20,000-$24,999 
0 $25,000-$29,999 
0 S30,000*$39,999 
0 $40,000*$49,999 
0 $50,000-$59,999 
0 $6O,OO0>$74,999 
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O $75,000-$99,999 
o $100,000-$! 49,999 
O $150,000-$199,999 
O $200,000-$249,999 
O $250,000 or more 


45. Would you describe your family as: (Hark one.) 


O Low income 
O Middle Income 
O Upper«middle income 
0 High income 


46. What did you do last summer (2006) that was particularly Important to you? 


47. Did your experience last summer (2006) advance your interest in studying engineering? 


48. Did you participate over the last summer (2006) in any of the following? (Mark all that apply.) 


O Engineering related InternshIp/Job 
d Engineering related research 
n Engineering related coursework 


49. What do you plan to do after graduating from college? 


O Ves 
O No 


□ N/A 
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SO. Did you take the Academic Pathways of People Learning Engineering Survey (APPLES)? 

O 

O No 

O 1'*” not sure 


Please click the SUBMIT SURVEY button below to access the next page of the survey, and ONLY after you have 
completed this page. Once you click the button, you will not be able to return to this page. 

Saw I Submit Survey | 


The survey title and other general fields are on the General tab. Individual survey questions are added and modified on the Questions tab. 
Questions may be re-ordered or deleted from the Order tab. You may see a preview of your survey at any time, by going to the Preview tab. If 
you have no further changes click finish to go back to the Management Interface. 

Click here to open the help window. 

^GenemlJ ^OuattlonsJ ^Orda^ ^^mviawj ^^mishj 
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p h p E S P 


Survey Design 


Help 

General Questions Order Preview Finish 


This is a preview of how this survey will look, In the preview the survey navigation buttons are inactive, use the section number buttons to view different 
sections, Some navigation buttons may not appea- on your final survey, depending on what access it is assigned. The survey will use the background color 
of the document in which it is embedded. If you have no further changes click Finish at the bottom of this page. 


PLEASE CLICK THE SUBMIT SURVEY BUTTON CNLY AFTER YOU HAVE COMPLETED THE SURVEY. FOR BEST VIEWING RESULTS, 
PLEASE HAXIHI2E YOUR BROWSER WINDOW. 


1, In the space provided, list 5 terms you would use to describe "engineering.” 


2. If you were asked the same question next week, how likely is it that you would list the same 5 terms? 


O Not likely 
O Somewhat likely 
G' Likely 
Very likely 
Q Extremely likely 


3, In the space provided, list 5 activities you think engineers do at work. 


Academic Pathways Study Seeing 2007 Survey 
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4. If you were asked the same question next week, how likely is it that you would list the same 5 activities? 

O Not liMy 

Somewhat likely 

o 

O Very Nkely 
^ Extremely likely 


S. In the space provided, list 5 terms you would use to describe "design.” 


6. If you were asked the same question next week, how likely is it that you would list the same S terms? 

O Not IliKly 
Q Somewhat likely 

O 

O Very likely 
O ExborTKily likely 


7. Of the twenty*three design activities below, please put a check mark next to the SIX MOST IMPORTANT. 

n AbetracUng 
n Bialnstormlng 
1*1 Building 
HI Communicating 
r~[ Decomposing 
ri Evaluatlitg 
I~1 Generating alternatives 
r~) Goal setting 
O IdenUfylrtg constraints 
r~| Imaginirrg 
O Iterating 
n Making decisions 
1~1 Makittg trade-offs 
n Modeling 
n Ptannmo 

□ Prototyping 

r~) Seeking Information 
n Sketching 

□ Synthesl/Mg 
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n Tesbng 

[~] Understanding die problem 
r~l Using creativity 
n Vl9uali2lng 


8. Of the twenty-three design ectivities below, please put a check marie next to the SIX LEAST IMPORTANT. 

PI Abstracting 
PI Bmlnstormbig 
□ Ouikling 
PI Communicating 
I~l Decomposing 
PI Evaluating 
PI Generating altematlves 
n Goal setting 
PI IdenUfyIng coeslralnts 
Q Imagining 
P) Iterating 
Q Making doclslons 
n Making trado-olfs 
PI Modeling 
n Planning 
n Prototyping 
PI Seeking information 
P Sketching 
PI Synthosirlng 
PI Testing 

PI Understandirrg Che problem 
PI Using creativity 
PI Visualizing 


9. For the following engineering design acbvitiet, please indicate your level of confidence. For example, if you have little or no 
confidence in your ability to model engineering solutions, then mark poor. If you are extremely confident in your ability, mark 
excellent. 


Poor 

DellniiKI what the problem really is O 
Searching for and collecting Information needed to solve the problem O 
Thinking up potential solutions to the problem Q 
Detailing how to buMd the solution to the problem ^ 
Aasesslrto and passing Judgment on a possible or planned solution to the problem Q 
Comparing and contrasting two soluUone to the problem on a particular dlmenston ^ 

such as cost 

Selecting one Idna or solution to the problem from among those considored Q 
Communicatliig elements ol the design in sketches, cliagiams, lists, and wiiticn or a 

oral reports 


Fair Good Eacellent 

Good 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 

o o o o 


10. For the following engineering design activities, please indicate how often you engaged in the activity in your coursework in 
the current academic year. 
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NewH 

1-2 

times 

1-2 

times 

Once 

2*3 

times 

Duly 


per 

term 

a 

rrxxith 

a 

week 

a 

wrjek 

Defining what Ihc probluiTi roally Is 

o 

o 

o 

o 

o 

o 

Searchirrg for and colloctlng Information riotxhxl to solve dm probkxn 

o 

o 

o 

o 

o 

o 

Tliinklng up potential aoludons Id the problem 

o 

o 

o 

o 

o 

o 

DrHallirrg how to build the solution lo the problem 

o 

o 

o 

o 

o 

o 

Assessing and passing Judgment on a possible or pkinrted solution to the 

problem 

o 

o 

o 

o 

o 

o 

Comparing and contrasting two soludona bo the probiem on a particular 
dimension such as cost 

o 

o 

o 

o 

o 

o 

Selecting one idtu or solution to the problem from aritong those considered 

o 

o 

o 

o 

o 

o 

Communicating elements of the dr»lgn In vfctrtches, diagrams, lists, artd wrideri 

or oral reports 

o 

o 

o 

o 

o 

o 


1 1 . For the fellewinfi •noincering design activitist, pleats indicate hew well you think your courses are preparing you to engage 
in the activity. For example, if you think they are not preparing you at all, then mark poor. If you think they are preparing 
you extremely well, then mark excellent. 


Poor Fair 


Wol Eicwllenl 


Deflnlrtg what the probleni really ts O 
Searching for and collecting Information needed to solve the problem ^ 
thinking up potendal solutions to the problem O 
Oetalllrtg how to buld the solution to the problem 
Amessirrg and passing Judgment on a possible or planned solution to the problem Q 
Comparing and contrasting two soluOonc to the problem on a particular dimension ^ 

such as cost ^ 

Selecting one Idea or solution lo tlie problem from among those oonsklerecl O 
Communicatliig clemonts of the design In sketches, diogiams, llstu, and written or a 

oial reports ^ 


o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 


13. You have been asked to dotign a playground. You havo a limitod amount of time and resources to gather information for 

your design. From the following list, please put a check mark ivaxt to tho FIVE kinds of information you would HOST LIKELY 
NEED as you work on your design: 

n Availability of materials 
n Body proportions 
[~1 Budget 

r~l HaiKlIcappod aconimlblllty 
r~l Information about the ar^ 
rn labor availability and cost 
n Logal liability 
n Maintenance concerns 
n Matnrlal costa 
n Material specifications 
n Neighborhood dt^mographlcs 
[~~1 Neighborhood opinions 
□ Safrsty 

n Supervision concerns 
n Technical references 
G UtllRlTfs 
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13. You have been aiked to desiffn a playground. You have a limited amount of time and reeources to gather information for 

your design. From the following list, please put a check mark next to the FIVE kinds of information you would LEAST LIKELY 
NEED as you work on your design: 

r? Avflilflbility of materials 
f~) Body proportions 

□ Budgftt 

f~1 Handicapped accessibility 
|~l Information about tlic area 
r~l labor availability and coat 
n Legal liability 
r*1 Maintenance concerrts 
D Material costs 
(~l Material specifications 
D Neighborhood demographics 
n Neighborhood opinions 
O Safety 

f~l Supervhclori concerns 
n Technical refererxres 

□ UtIWIes 


14. Of the 20 items below, please put a check mark next to the FIVE you think are MOST IMPORTANT to precticirrg engineers. 

r~1 Ehrslnew krtowtedge 
n Communication 
n Conducting eiperiments 
n Contemporary Iscuos 
f~l Creativity 
r~| Data analysis 
f~1 Design 

r~l Errglnoorlng analysis 
n Englnoering tools 
n Ethics 
n Global context 
n Leadership 
r~| Life-long InamirKi 
r~] Management ^lls 
□ Math 

n Problem solving 
r*] Professionalism 
n SclerKe 
n Societal context 
n Teamwork 


IS. Please rate hew well prepared you are to incorporate each of the followirvg items while practicing as an engineer; 



Not at 


Somewhat 


Very 

well 


prepared 


prepared 


prRfNirtxl 

Business krrowledge 

o 

o 

o 

o 

o 

Communication 

o 

o 

o 

o 

o 

Conducting experiments 

o 

o 

o 

o 

o 

Contemporary issues 

o 

o 

o 

o 

o 
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Creativity 

o 

o 

o 

o 

o 

Data analysis 

o 

o 

o 

o 

o 

Design 

o 

o 

o 

o 

o 

fngtneering analysis 

o 

o 

o 

o 

o 

Engineering tools 

o 

o 

o 

o 

o 

Ethics 

o 

o 

o 

o 

o 

Gtobal context 

o 

o 

o 

o 

o 

leadership 

o 

o 

o 

o 

o 

Life- long learning 

o 

o 

o 

o 

o 

Management skilb 

o 

o 

o 

o 

o 

Mall) 

o 

o 

o 

o 

o 

Problem soivkig 

o 

o 

o 

o 

o 

Frofoasionallsm 

o 

o 

o 

o 

o 

SclOTKO 

o 

o 

o 

o 

o 

Societal context 

o 

o 

o 

o 

o 

Teamwork 

o 

o 

o 

o 

o 


16. You aro an enginaar working for a silicon chip manufacturing company. Your company's currant facilitias in California ara 

ciosa to maximum capacity and tha company is out of land to axpand at tha currant sita. Tha company naads to build a now 
factory in a now location. Thara ara thraa potantial sitas: in tha U.S. in Alabama or Illinois; or in Asia in Thailand. You hava 
baan askad to avaluata tha sitas for locating tha plant. 

Plaasa list FIVE factors you think would ba important in your avaluation of tha sitas. 


17. Please rate how well prepared you are to consider each of the following items as a pracbcing engineer: 



Not at 
all 

prepared 


Somewhat 

prepared 


Very 

well 

prepared 

Economic context 

o 

o 

o 

o 

o 

Global context 

o 

o 

o 

o 

o 

Natural context 

o 

o 

o 

o 

o 

Societal context 

o 

o 

o 

o 

o 

Tcchnotoglcal context 

o 

o 

o 

o 

o 


18. During your undergraduate years, how many months of experience have you had with academic research or professional 
engineering? 


mom 

None 

months months months months months 10 

months 


Part-time, academic reaearch work on campus Q ^ 

full-Nmo, acadnmic reanarch work on campus O O 


o 

o 


o 

o 


o o o 

o o o 
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Part Ume engineering work (e.g., internship, coop, summer Job) ^ ^ 

Full-time engineering work (e.g.. internship, coop, summer job) O O 


o 

o 


o 

o 


o o o 

o o o 


19. Please provide a short description of your expcrience(s) with academic research and/or professional engineering. 


ttSorvey | 


The survey title arxl other gerteial fields are on the General tab. IrKlIvIdual survey questions are added and rrtodMIed on the Questions tab. QuesUorts may be 
re ordered or deleted from the Order tab. You may see a preview of your survey at any Ume. by goirrg to the Preview tab. D you have no further changes click 

Finish to go bock to the Management Interface. 
click here to open the H«Hp wlrxlow. 

General | Quetiionc | Order | Preview | Fmh | 
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IVariable and Item Content 
1a: Academic Persistence 


1b: Professionai Persistence 


2a: Motivation (Financiai) 


1 SURV-1:C1Y1A Winter 04 | SURV 2: CIYIB Spring 04 | SURV 3: C1Y2A Fall 04 | 

23. Do see yourself studying or practicing ei25. Do see yourself studying or p2. Do you intend to complete a maj 

2. What do you intend to major in? 2. What do you intend to major ir3. What do you intend to major in? 

3. What do you intend to major in (currently 3. What do you intend to major ir4. If you intend to double major, wh 

4. What is the highest academic degree tha 4. What is the highest academic degree that you intend to obtain in e 

Not asked Not asked 5. Do you intend to practice, condu 


7c. Engineers make more money than most 8c. Engineers make more monev7c. Engineers make more money tl 
7g. Engineers are well paid 8g. Engineers are well paid 7f. Engineers are well paid 

7j. An engineering degree will guarantee me8j. An engineering degree will gu 7i. An engineering degree will guar 
7h. Engineering is an occupation that is resf 8h. Engineering is an occupation 7g. Engineering is an occupation tf 


2b: Motivation (Famiiy infiuence) 


7d. My parents would disapprove if 


2c: Motivation (Sociai Good) 


2d: Motivation (High Schooi Mentor infiuence) 


2e: Motivation (Mentor infiuence) 


3a: Confidence in Math & Science Skiiis 


3b: Confidence in Professionai and 
interpersonai Skiiis 


3c: Confidence in Soiving Open-ended 
Probiems 


7i. My parents want me to be an engineer 8i. My parents want me to be an 7h. My parents want me to be an e 
7d. My parent(s) are making me study engirSd. My parent(s) are making me study engineering 

7b. Technoiogy piays an important roie in sc 8b. Technoiogy piays an importa 7b. Technoiogy piays an important 

7f. Engineers have contributed greatiy to fixi8f. Engineers have contributed gi7e. Engineers have contributed gre 


Not asked 
Not asked 
Not asked 


8i. High schooi teachers/advisors Not asked 

9k. I had one or more high schooi math/science teachers who seeme 

9n. One or more of my favorite high schooi teachers were math/sciei 


Not asked 


Not asked 


7k. A facuity member, academic ac 


Not asked 


Not asked 


7i. A non-university affiiiated mentc 


19f. Math abiiity 
19g. Science abiiity 


19b. Seif confidence (sociai) 

19d. Leadership abiiity 
19e. Pubiic speaking abiiity 
19i. (Written) Communication skiiis 
19j. Business abiiity 

19c. Seif understanding 


21 f. Math abiiity 
21 g. Science abiiity 


21b. Seif confidence (sociai) 

21 d. Leadership abiiity 
21 e. Pubiic speaking abiiity 
21 i. (Written) Communication ski 
21j. Business abiiity 

21c. Seif understanding 


lOe. Math abiiity 
lOf. Science abiiity 

lOj. Abiiity to appiy math & science 

10a. Seif confidence (sociai) 

10c. Leadership abiiity 
lOd. Pubiic speaking abiiity 
lOh. Communication skiiis 
10k. Business abiiity 
lOi. Abiiity to perform in teams 
10b. Seif understanding 

9a. Creative thinking is one of my s 


8g. I enjoy probiems that can be soived in d 9h. I enjoy probiems that can be 9d. I am skiiied at soiving probiems 


1 0b. Confidence: Critical thinking skills 1 1 b. Confidence: Critical thinking 1 0i. Critical thinking skills 

7j. Engineers are creative problem 
14b. Since the beginning of fall terr 

4a: Perceived Importance in Math & Science 

Skills 20b. Math ability 22b. Math ability 1 1a. Math ability 

20c. Science ability 22c. Science ability 11b. Science ability 

Ilf. Ability to apply math & science 

4b: Perceived Importance in Professional & 

Interpersonal Skills 


20a. Public speaking ability 
20e. (Written) Communication skills 
20f. Business ability 


22a. Public speaking ability 

22e. (Written) Communication sk 1 1d. Communication skills 
22f. Business ability 1 1g. Business ability 

1 1 h. Ability to perform in teams 


5: Knowledge of the Engineering Profession 


9q. I am familiar with what a pracOc. I am familiar with what a practic 


33. Did any of your immediate family membi26. How many of your friends and family members are practicing en( 
27. What portion of your friends in college (c27. What portion of your friends in college (on this campus or other c 


22b. Teaching methods - Individual projects 24b. Teaching methods - Individi19b. Teaching methods - Individua 


6a: Exposure to Project-Based Learning 
Methods: Individual 

6b: Exposure to Project-Based Learning 
Methods: Team 
7: Collaborative Work Style 


8: Extracurricular Fulfillment (Non- 
engineering) 


8b: Extracurricular Involvement (Engineering) 

8c: Research Experience 
9: Curriculum Overload 


10: Financial Difficulties 

11a: Academic Disengagement - Liberal Arts 
courses 


11b: Academic Disengagement - Engineering- 
Related courses 


22c. Teaching methods - Team projects 24c. Teaching methods - Team p 1 9c. Teaching methods - Team prc 
8a. I prefer working/studying alone (reverse 9a. I prefer working/studying alorSa. I prefer studying in a group to s 

8c. I prefer working as part of a tea 
8d. I get along well with others in si 

21 . Importance of non-engineering 

22. Frequency of involvement in nc 


23a. Pressure - course load 
23b. Pressure - course pace 

23c. Pressure - Balance between s 


24. How well are you meeting work 

25. How stressed do you feel in yoi 


38. Do you have any concerns about your ability to finance your college educ26. Do you have any concerns abo 
13e. Frequency: Worried about how you would pay for school 


17a. Came late to liberal arts class 
17b. Skipped liberal arts class 


1 8a. Came late to liberal arts da: 1 6a. Came late to non-engineering 
18b. Skipped liberal arts class 16b. Skipped non-engineering clas 


17c. Turned in liberal arts assignments that 18c. Turned in liberal arts assign 16c. Turned in non-engineering as: 
17d. Turned in liberal arts assignments late 18d. Turned in liberal arts assign 16d. Turned in non-engineering asi 
17e. Thought liberal arts classes were borin 18e. Thought liberal arts classes 16e. Thought non-engineering clas 


1 6a. Came late to engineering class 1 7a. Came late to engineering cl; 1 5a. Came late to engineering clas 

16b. Skipped engineering class 17b. Skipped engineering class 15b. Skipped engineering class 


16c. Turned in engineering assignments tha17c. Turned in engineering assig 15c. Turned in engineering assignr 
16d. Turned in engineering assignments lat(17d. Turned in engineering assig 15d. Turned in engineering assignr 


11c: Academic Disengagement - Overaii 
12a: Frequency of interaction with Facuity 

12b: Frequency of interaction with Teaching 
Assistants 

12c: Frequency of interaction with instructors 

13a: Satisfaction with Facuity 

13b: Satisfaction with Teaching Assistants 

13c: Satisfaction with instructors 

13d. Satisfaction with advising 

14: Satisfaction with Academic Faciiities 

15: Overaii Satisfaction with Coiiegiate 
Experience 

16: intrinsic Motivation (Psychoiogicai) 


1 6e. Thought engineering ciasses were bori 1 7e. Thought engineering ciasse 1 5e. Thought engineering ciasses ' 
Constructs 1 1 a + 1 1b Constructs 1 la + 1 1b Constructs 1 1 a + 1 1 b 

1 8a. Facuity during office hours 20a. Facuity during office hours 1 7a. Facuity during office hours 

1 8b. Facuity outside of ciass or office hours 20b. Facuity outside of ciass or c 1 7b. Facuity outside of ciass or offi 


1 8c. TAs during office hours 20c. TAs during office hours 1 7c. TAs during office hours 

1 8d. TAs outside of ciass or office hours 20d. TAs outside of ciass or offic 1 7d. TAs outside of ciass or office I 


Constructs 12a + 12b 


Constructs 12a + 12b 


Constructs 12a + 12b 


1 1 b. Quaiity of instruction by facuity 


12b. Quaiity of instruction by fact 12a. Quaiity of instruction by facuit; 


11c. Avaiiabiiity of facuity 
lid. Quaiity of instruction by TAs 


1 2c. Avaiiabiiity of facuity 1 2b. Avaiiabiiity of facuity 

12d. Quaiity of instruction by TAs 12c. Quaiity of instruction by TAs 


1 1 e. Availability of TAs 

Constructs 13a + 13b 


12e. Availability of TAs 
Constructs 13a + 13b 


12d. Availability of TAs 

Constructs 13a + 13b 


12a. Computer facilities 
12b. Libraries 
12c. Classrooms 
12f. Laboratories 


13a. Computer facilities 
13b. Libraries 
13c. Classrooms 
13f. Laboratories 


13a. Computer facilities 
13b. Libraries 
13c. Classrooms 
13f. Laboratories 


6. Qverall quality of collegiate ex|35. Qverall quality of collegiate exp 

29. I feel good when I am doing 
engineering activities 
29. Majoring in engineering is fun 
29. I think engineering is 
interesting 


16: Intrinsic Motivation (Behavioral) 


Demographic Variables 


Expected Graduation Date 1 . Expected year of graduation 1 . Expected year of graduation 

Expected GPA academic term 
Expected GPA overall 

Sex 24. Sex 

Age 25. How oid wiii you be on December 31st of this year? 

Ethnicity 26. Ethnic background 
Marital status 28. Maritai status 
Dependents 29. Number of dependents 
Citzenship 27. Citizenship 


1 . Expected year of graduation 


Current academic standing 
T raditional/nontraditional student 

Preliminary area of interest 
Full time/part-time student 
Year graduated from high school 
High school community 
Average grade in high school 

Residence in college 
Roommates in college 
Disabilities 
Assistive technologies 
Cultural Background 

Cultural Background 
Cultural Background 
Cultural Background 
SES 
SES 
SES 
SES 
SES 
SES 
SES 


30. What year did you graduate from high schooi? 

31 . How wouid you describe the community where you attended high schooi? 

32. What was your average grade in high schooi? 

33. Where are you iiving now whiie attending coiiege? 

34. With whom do you iive during the schooi year? 

41 . Do you have any of the foiiowing physicai, iearning, or heaith disabiiities? 

42. Do you receive any of the foiiowing assistive technoiogy or other accommodations? 


37a. How weii do you meet your coiiege expenses - seif (income) 
37b. Coiiege expenses - seif (savings) 

37c. Coiiege expenses - parents and famiiy 
37d. Coiiege expense - empioyer support 
37e. Coiiege expenses - schoiarships and grants 
37f. Coiiege expenses - ioans 
37g. Coiiege expenses - other sources 


27a. How weii do you meet your cc 
27b. Coiiege expenses - seif (savin 
27c. Coiiege expenses - parents ar 
27d. Coiiege expense - empioyer s 
27e. Coiiege expenses - schoiarsh 
27f. Coiiege expenses - ioans 


SES 

SES 

SES 

SES 


34. Highest level of education of mother 

35. Highest level education of father 

36. Best estimate of parents' total income 


Unofficial item groupings 

Research 

Research 

Research 

Engineering Extra-curricular 
Engineering Extra-curricular 
Engineering Extra-curricular 
Engineering Extra-curricular 


34b. Did you participate in enginee 


34a. Did you participate in an engir 
34c. Did you participate in engineei 
52. What are your summer plans 32. What did you do this summer ti 

33. Did your experience advance yi 


Engineering Extra-curricular 


Engineering Extra-curricular 


Engineering 

Engineering 


Extra-curricular 

Extra-curricular 50. Some students participate in design competitions, internships, and clubs. In the space provided identify engir 
NSSE 6a. Hours spent - preparing for class 7a. Hours spent - preparing for cl 31 a. Hours spent - preparing for cl< 
NSSE 6b. Hours spent - working for pay 7b. Hours spent - working for pa^ 31 b. Hours spent - working for pay 

NSSE 6c. Hours spent - participating in co-curriculi7c. Hours spent - participating in 31c. Hours spent - participating in ( 
NSSE 6d. Hours spent - relaxing and socializing 7d. Hours spent - relaxing and sc31d. Hours spent - relaxing and so( 
NSSE 6e. Hours spent - providing care 7e. Hours spent - providing care 31 e. Hours spent - providing care 

NSSE 6f. Hours spent - commuting to class 7f. Hours spent - commuting to c 31 f. Hours spent - commuting to cli 
Confirm/Doubt 
Confirm/Doubt 
Confirm/Doubt 

Confirm/Doubt 54. Identify any classes (in high school or cc43. Identify any classes this year that have STRONGLY REINFORC 
Confirm/Doubt 55. Identify any classes (in high school or cc44. Identify any classes this year that have STRONGLY WEAKENEt 


Survey verification 
Survey process information 

Survey process information 

Open-ended question 

items not assigned to a construct and Deieted 
items/constructs 


Grad schooi not in engineering 5. If you are thinking of going to graduate sc 5. If you are thinking of going to c 6. If you are thinking of going to grs 
Course taught by grad students 21 . What portion of the courses you have ta 23. What portion of the courses v 1 8. What portion of the courses yoi 


Teaching methods - iectures 
Teaching methods - iabs 
Teaching methods - seminars 
Competitive work styie (personai) 
Competitive work styie (personai) 
Competitive work styie (institutionai) 
Competitive work styie (institutionai) 
Competitive work styie (institutionai) 
Competitive work styie (institutionai) 
Coiiaborative work styie (institutionai) 
Coiiaborative work styie (institutionai) 
Coiiaborative work styie (institutionai) 
Coiiaborative work styie (institutionai) 
Satisfaction with academic services 
Satisfaction with academic services 
Exposure to PBL Methods 
Exposure to PBL Methods 
Exposure to PBL Methods 
Who Am I questions 
Confidence in computer skiiis 
Perceived importance of computer skiiis 


22a. During the current school year, what pc 24a. During the current school ye 1 9a. During the current school yeai 

22d. Teaching methods - Labs 24d. Teaching methods - Labs 19d. Teaching methods - Labs 

22e. Teaching methods - Seminars 24e. Teaching methods - Semine 19e. Teaching methods - Seminars 

8b. I am a competitive person 
8e. I strive to get higher grades tha 
8g. The educational institution I am 

8h. My instructors often remind students tha9i. My instructors often remind st 8h. My instructors often remind stu 

8f. I prefer keeping good ideas to rr 

8k. My instructors grade on a curve 9m. My instructors grade on a cu 8k. My instructors grade on a curve 

8i. I have easy access to work spaces wheri 9j. I have easy access to work sp 8i. I have easy access to work spac 
8j. I am encouraged by my instructors to init9l. I am encouraged by my instru 8j. I am encouraged by my instruct 

8I. The educational institution I am 
14a. Since the beginning of fall terr 

12d. Satisfaction with tutoring 13d. Satisfaction with tutoring 13d. Satisfaction with tutoring 

12e. Satisfaction with academic advising 13e. Satisfaction with academic ; 13e. Satisfaction with academic ad 

14c. Since the beginning of fall terr 
20. To what extent have your cours 

8e. I have strong problem solving skills 9e. I have strong problem solving 9b. I have strong problem solving s 

45.-51 . Who am I questions 

1 9h. Confidence: Computer and programmii 21 h. Confidence: Computer and 1 0g. Confidence: Computer skills 
20d. Perceived importance: Computer and f22d. Perceived importance: Com 11c. Perceived importance: Compi 


Perceived importance of criticai thinking skiiis 


lie. Perceived importance: Critical 


UPri questions - Motivation, Seif-Esteem, 

Ethnic identity 
Motivation (Beiief That Engineers improve 
Weifare of Society Through Creative Work) 
Motivation (Beiief That Engineers improve 
Weifare of Society Through Creative Work) 
Motivation (Beiief That Engineers improve 
Weifare of Society Through Creative Work) 


29.-32. UPri questions 

7a. I enjoy figuring out how things work 8a. I enjoy figuring out how thing; 7a. I enjoy figuring out how things v 
7e. Engineers are innovative 8e. Engineers are innovative 

7k. Engineers are creative 8k. Engineers are creative 

8b. I enjoy the subjects of science and math 9b. I enjoy the subjects of science and math the most 

8c. Creative thinking is one of my strengths 9c. Creative thinking is one of my strengths 

8d. Studying in a group is better than studyii9d. Studying in a group is better than studying by myseif 

8f. I enjoy taking iiberai arts courses more tf 9f. I enjoy taking iiberai arts courses more than science and math co 

9g. My friends are supportive of me when I am academicaiiy success 
9o. I can count on my friends at schooi for emotionai support when I 
9p. I can count on my friends at schooi for emotionai support when I 
9a. Importance: Getting higher grades than 10a. Importance: Getting higher grades than my ciassmates 
9b. Importance: Influencing social values 10b. Importance: Influencing social values 
9c. Importance: Becoming an authority in m 10c. Importance: Becoming an authority in my field 
9d. Importance: Keeping goood ideas to my lOd. Importance: Keeping goood ideas to myself unless it is to my ac 
9e. Importance: Helping to promote racial uilOe. Importance: Helping to promote racial understanding 
9f. Importance: Becoming a community leac lOf. Importance: Becoming a community leader 
9g. Importance: Helping others who are in d lOg. Importance: Helping others who are in difficulty 
9h. Importance: When playing any game, pi 1 0h. Importance: When playing any game, playing to win 
9i. Importance: Developing a meaningful ph lOi. Importance: Developing a meaningful philosophy of life 
9j. Importance: Becoming a practicing engin lOj. Importance: Becoming a practicing engineer 
9k. Importance: Getting along with others 10k. Importance: Getting along with others 

91. Importance: Working as part of a team 1 01. Importance: Working as part of a team 

9m. Importance: Becoming a student goverr 10m. Importance: Becoming a student government official 
9n. Importance: Establishing relationships v\ lOn. Importance: Establishing relationships with engineering student 
9o. Importance: Establishing relationships v\ lOo. Importance: Establishing relationships with non-engineering stu 
10a. Confidence: Analytical and problem so 11a. Confidence: Analytical and problem solving skills 
10c. Confidence: General knowledge 11c. Confidence: General knowledge 
1 0d. Confidence: Knowledge of a particular 1 1 d. Confidence: Knowledge of a particular field or discipline 
lOe. Confidence: Interest in studying engine lie. Confidence: Interest in studying engineering 
11a. Satisfaction: Opportunities for common 12a. Satisfaction: Opportunities for community service 
13a. Frequency: Felt that your courses inspi 14a. Frequency: Felt that your courses inspired you to think in new v 
13b. Frequency: Felt you did not have enou 14b. Frequency: Felt you did not have enough time to pursue nonac; 
1 3c. Frequency: Worried about keeping up 1 1 4c. Frequency: Worried about keeping up with your schoolwork 
13d. Frequency: Felt you did not have a "so 14d. Frequency: Felt you did not have a "social life" 

13f. Frequency: Felt stressed 14e. Frequency: Felt stressed 


13g. Frequency: Participated in a peer stud^ 14f. Frequency: Participated in a peer study group 

14g. Frequency: Visited or worked in a commerciai engineering envii 
14h. Frequency: Sociaiized with someone of another raciai/ethnic gn 
14i. Frequency: Discussed raciai issues 

14j. Frequency: Attended a raciai/cuiturai awareness workshop/even 
14a. Came iate to math ciass 15a. Came iate to math ciass 

14b. Skipped math ciass 15b. Skipped math ciass 

14c. Turned in math assignments that did n( 15c. Turned in math assignments that did not refiect your best work 

14d. Turned in math assignments iate 15d. Turned in math assignments iate 
14e. Thought math ciasses were boring 15e. Thought math ciasses were boring 
1 5a. Came iate to science ciass 1 6a. Came iate to science ciass 

1 5b. Skipped science ciass 1 6b. Skipped science ciass 

15c. Turned in science assignments that die 16c. Turned in science assignments that did not refiect your best wo 

15d. Turned in science assignments iate 16d. Turned in science assignments iate 
1 5e. Thought science ciasses were boring 1 6e. Thought science ciasses were boring 

19a. Frequency: Asked for advice about managing your coursework 
19b. Frequency: Worked with peopie who have diverse backgrounds 
19c. Frequency: Spent time in a non-academic roie 
19d. Frequency: Requested feedback on course assignments from a 
1 9e. Frequency: "Crammed" aii night studying for an exam or compif 
1 9f. Frequency: Worked coiiaborativeiy on an assignment that was p 
19g. Frequency: Decided to turn in "C" quaiity work over spending cc 
19h. Frequency: Flad a research experience on a facuity or graduate 
19i. Frequency: Prioritized a good grade in a generai education cour 
19j. Frequency: Prioritized a good grade in a math, science, or engin 
19k. Frequency: Studied reguiariy in biocks of 2 hours or more 
1 9i. Frequency: Took a seminar course to discuss and argue ideas v 
1 9m. Frequency: Reduced time spent on course work to have more 
19n. Frequency: Asked for advice about managing your coiiege expe 
1 9o. Frequency: Created a project outside of your academic work wi 
1 9p. Frequency: Monitored how you spent your time on your course 
1 9q. Frequency: Asked for heip to strengthen a particuiar skiii (e.g., i 
lOr. Frequency: Reduced your course ioad to improve your grades 
19a. Confidence: Seif-confidence (inteiiectu 21a. Confidence: Seif-confidence (inteiiectuai) 

52a. Frequency: Worked on ciass projects 28a. Frequency: Worked on ciass projects 
52b. Frequency: Fleid a study group 28b. Frequency: Fleid a study group 

52c. Frequency: Took a specific iectu re-type 28c. Frequency: Took a specific iecture-type ciass 
52d. Frequency: Took a specific iaboratory (28d. Frequency: Took a specific iaboratory ciass 
52e. Frequency: Worked on homework 28e. Frequency: Worked on homework 
52f. Frequency: Reviewed ciass materiai 28f. Frequency: Reviewed ciass materiai 



52g. Frequency: Prepared for class exams 28g. Frequency: Prepared for class exams 
52h. Frequency: Wrote class reports 28h. Frequency: Wrote class reports 

53. In what ways do you Interact with other (39. In what ways do you Interact with other engineering students out 

40. When and under what circumstances (If any) do you rely on you 

41 . Do you feel uncomfortable seeking emotional support from your 1 

42. What Is the race/ethnIcIty of your six closest friends at school? (c 

43-46. Phil Bell's technology questions 

48. What Is the first word or phrase you think of to describe your favorite professor? 

49. Do you believe your peers would agree with this description? 

56. What Intellectual, personal, financial, and other challenges do you feel you may need to overcome Ito gradua 



SURV 4: C1Y2B Spring 05 | SURV 5: C1Y3A Fall 05 | SURV 6: C1Y3B Spring 06 | 

2. Do you intend to complete a rr2. Do you intend to complete a major ii2. Do you intend to complete a major in engineering? 

3. What do you intend to major ir 3. What do you intend to major in? 3. What do you intend to major in? 

4. If you intend to double major, \4. If you intend to double major, what i 4. If you intend to double major, what is the second maj 
jngineering? 


5. Do you intend to practice, con( 5. Do you intend to practice, conduct n 5. Do you intend to practice, conduct research in, or tea 


7b. Engineers make more mone^ 7b. Engineers make more money than 7b. Engineers make more money than most other profe 
7e. Engineers are well paid 7e. Engineers are well paid 7e. Engineers are well paid 

7h. An engineering degree will gi 7h. An engineering degree will guaran' 7h. An engineering degree will guarantee me a job whe 
7f. Engineering is an occupation 7f. Engineering is an occupation that is7f. Engineering is an occupation that is respected by otf 


7c. My parents would disapprove 7c. My parents would disapprove if I d 7c. My parents would disapprove if I chose a major othe 



7g. My parents want me to be an 7g. My parents want me to be an engir 7g. My parents want me to be an engineer 

7a. Technology plays an importa 7a. Technology plays an important rol(7a. Technology plays an important role in solving sociel 
7d. Engineers have contributed c7d. Engineers have contributed greatly 7d. Engineers have contributed greatly to fixing problerr 


Not asked Not asked Not asked 

3d genuinely excited about math/science 
nee teachers 


7j. A faculty member, academic £ 7j. A faculty member, academic advsic 7j. A faculty member, academic advsior, teaching assist 


7k. A non-university affiliated me 7k. A non-university affiliated mentor h 7k. A non-university affiliated mentor has encouraged a 


10d. Math ability 9d. Math ability 

10e. Science ability 9e. Science ability 


1 0d. Math ability 
lOe. Science ability 


1 0h. Ability to apply math & scier 9h. Ability to apply math & science prir 1 0g. Ability to apply math & science principles in solvinc 


10a. Self confidence (social) 
10b. Leadership ability 
10c. Public speaking ability 
lOg. Communication skills 
lOi. Business ability 
lOj. Ability to perform in teams 


9a. Self confidence (social) 
9b. Leadership ability 
9c. Public speaking ability 
9g. Communication skills 
9i. Business ability 
9j. Ability to perform in teams 


10a. Self confidence (social) 
10b. Leadership ability 
10c. Public speaking ability 
lOf. Communication skills 
lOh. Business ability 
1 0i. Ability to perform in teams 


9a. Creative thinking is one of m^ 8e. Creative thinking is one of my strer 9a. Creative thinking is one of my strengths 


9c. I am skilled at solving probler 8g. I am skilled at solving problems the 9c. I am skilled at solving problems that can have multif 



10k. Critical thinking skills 10k. Critical thinking skills lOj. Critical thinking skills 

71. Engineers are creative problei7i. Engineers are creative problem sol' 71. Engineers are creative problem solvers. 

n, how often you felt challenged to solve open-ended problems that might have multiple solutions 

1 1 d. Math ability 1 0d. Math ability 1 1 d. Math ability 

1 1 e. Science ability 1 0e. Science ability 1 1 e. Science ability 

1 1 g. Ability to apply math & scier 1 0h. Ability to apply math & science pr 1 1 g. Ability to apply math & science principles in solvinc 

1 1 a. Self confidence (social) 1 0a. Self confidence (social) 1 1 a. Self confidence (social) 

1 1 b. Leadership ability 1 0b. Leadership ability 1 1 b. Leadership ability 

1 1 c. Public speaking ability 1 0c. Public speaking ability 1 1 c. Public speaking ability 

1 1 g. Communication skills 1 0g. Communication skills 1 1 f. Communication skills 

1 1 i. Business ability 1 0i. Business ability 1 1 h. Business ability 

1 1j. Ability to perform in teams 1 0j. Ability to perform in teams 1 1 i. Ability to perform in teams 

9b. I am familiar with what a prac 8f. I am familiar with what a practicing 9b. I am familiar with what a practicing engineer does 


30. How much exposure have yo29. How much exposure have you had 30. How much exposure have you had to a professiona 

28. Do you have any family mem 27. Do you have any family members '28. Do you have any family members who are practicinc 

29. Do you have any close friend 28. Do you have any close friends whc29. Do you have any close friends who are practicing er 



19b. Teaching methods - Individi 18b. Teaching methods - individuai prt 19b. Teaching methods - individuai projects 

19c. Teaching methods - Team p 18c. Teaching methods - Team projeci 19c. Teaching methods - Team projects 
8a. i prefer studying in a group tc 8a. i prefer studying in a group to stud; 8a. i prefer studying in a group to studying by myseif 
8c. i prefer working as part of a t( 8b. i prefer working as part of a team t 8c. i prefer working as part of a team to working aione 
8d. i get aiong weii with others in 8c. i get aiong weii with others in studv 8d. i get aiong weii with others in study situations 
8j. i am a coiiaborative person 8d. i am a coiiaborative person 8j. i am a coiiaborative person 

21 . importance of non-engineerir 20. importance of non-engineering act 21 . importance of non-engineering activities 

22. Frequency of invoivement in 21 . Frequency of invoivement in non-e 22. Frequency of invoivement in non-engineering activit 


36. Research experiences since coming to coiiege 
23a. Pressure - course ioad 22a. Pressure - course ioad 23a. Pressure - course ioad 

23b. Pressure - course pace 22b. Pressure - course pace 23b. Pressure - course pace 

23c. Pressure - Baiance betweer22c. Pressure - Baiance between socii23c. Pressure - Baiance between sociai and academic I 

24. How weii are you meeting wc 23. How weii are you meeting workioa( 24. How weii are you meeting workioad demands of yoi 

25. How stressed do you feei in ) 24. How stressed do you feei in your c 25. How stressed do you feei in your coursework right r 

26. Do you have any concerns ai 25. Do you have any concerns about y 26. Do you have any concerns about your abiiity to final 


1 6a. Came iate to iiberai arts da; 1 5a. Came iate to iiberai arts ciass 1 6a. Came iate to iiberai arts ciass 

1 6b. Skipped iiberai arts ciass 1 5b. Skipped iiberai arts ciass 16b. Skipped iiberai arts ciass 

16c. Turned in iiberai arts assign 15c. Turned in iiberai arts assignments 16c. Turned in iiberai arts assignments that did not refie 

16d. Turned in iiberai arts assign 15d. Turned in iiberai arts assignments 16d. Turned in iiberai arts assignments iate 

16e. Thought iiberai arts ciasses 15e. Thought iiberai arts ciasses were 16e. Thought iiberai arts ciasses were boring 

15a. Came iate to engineering ci; 14a. Came iate to engineering ciass 15a. Came iate to engineering ciass 
1 5b. Skipped engineering ciass 1 4b. Skipped engineering ciass 1 5b. Skipped engineering ciass 

1 5c. Turned in engineering assig 14c. Turned in engineering assignmen 15c. Turned in engineering assignments that did not ref 
15d. Turned in engineering assig 14d. Turned in engineering assignmen 15d. Turned in engineering assignments iate 



15e. Thought engineering ciasse 14e. Thought engineering ciasses wer 15e. Thought engineering ciasses were boring 
Constructs 1 1a + 11b Constructs 1 la + 1 1b Constructs 1 1 a + 1 1 b 


17a. Facuity during ciass 
17b. Facuity during office hours 


1 6a. Facuity during ciass 
16b. Facuity during office hours 


17a. Facuity during ciass 
17b. Facuity during office hours 


17c. Facuity outside of ciass or o 16c. Facuity outside of ciass or office f 17c. Facuity outside of ciass or office hours 


17d. TAs during ciass 
17e. TAs during office hours 


1 6d. TAs during ciass 
1 6e. TAs during office hours 


17d. TAs during ciass 
17e. TAs during office hours 


17f. TAs outside of ciass or office 16f. TAs outside of ciass or office hour 17f. TAs outside of ciass or office hours 


Constructs 12a + 12b 


Constructs 12a + 12b 


Constructs 12a + 12b 


12a. Quaiity of instruction by fact 
12b.Quaiity of advising by facuity 
1 2c. Avaiiabiiity of facuity 
12d. Quaiity of instruction by TAs 
12e. Quaiity of advising by TAs 
12f. Availability of TAs 

Constructs 1 3a + 1 3b 


11a. Quality of instruction by faculty 
1 1 b. Quality of advising by faculty 
1 1c. Availability of faculty 
lid. Quality of instruction by TAs 
1 1 e. Quality of advising by TAs 
Ilf. Availability of TAs 

Constructs 13a + 13b 


12a. Quality of instruction by faculty 
1 2b. Quality of advising by faculty 
1 2c. Availability of faculty 
1 2d. Quality of instruction by TAs 
1 2e. Quality of advising by TAs 
12f. Availability of TAs 

Constructs 13a + 13b 


13a. Computer facilities 
13b. Libraries 
13c. Classrooms 
1 3f. Laboratories 


12a. Computer facilities 
12b. Libraries 
12c. Classrooms 
12f. Laboratories 


13a. Computer facilities 
13b. Libraries 
13c. Classrooms 
1 3f. Laboratories 


32. Qverall quality of collegiate e 31 . Qverall quality of collegiate experie 32. Qverall quality of collegiate experience 


38. I feel good when I am doing 

engineering activities 

38. Majoring in engineering is fun 


29. I think engineering is interesting 



1 . Expected year of graduation 1 . Expected year of graduation 


1 . Expected year of graduation 

51 . What is your expected GPA this academic term 


33. Sex 

34. Ethnic background 


35. Citizenship 


36. Average grade in H.S. 


27a. How weii do you meet your 26a. How weii do you meet your coiiec 27a. How weii do you meet your coiiege expenses - sei 
27b. Coiiege expenses - seif (sa\26b. Coiiege expenses - seif (savings) 27b. Coiiege expenses - seif (savings) 

27c. Coiiege expenses - parents 26c. Coiiege expenses - parents and fi27c. Coiiege expenses - parents and famiiy 

27d. Coiiege expense - empioyei26d. Coiiege expense - empioyer supp27d. Coiiege expense - empioyer support 

27e. Coiiege expenses - schoiar:26e. Coiiege expenses - schoiarships 27e. Coiiege expenses - schoiarships and grants 

27f. Coiiege expenses - ioans 26f. Coiiege expenses - ioans 27f. Coiiege expenses - ioans 



37. Highest level of education of mother 


32. Best estimate of parents' total income 


ring-related research last summei38b. Did you participate in engineering-related research last summer? 


leering related internship/job last 38a. Did you participate in an engineering related internship/job last summer? 
ring-related coursework last sumr38c. Did you participate in engineering-related coursework last summer? 

53. What are your summer plans 36. What did you do this summer that ' 52. What are your summer plans? (open) 
our interest in studying engineerir37. Did your experience advance your interest in studying engineering? (Y/N) 


leering-related activities you have 33. Reasons for a leave of absence during fall term 

31 a. Hours spent - preparing for 30a. Hours spent - preparing for class 31 a. Hours spent - preparing for class 

31b. Hours spent - working for ps 30b. Hours spent - working for pay 31b. Hours spent - working for pay 

31c. Hours spent - participating ii30c. Hours spent - participating in co-c31c. Hours spent - participating in co-curricular activitie: 

31 d. Hours spent - relaxing and £ 30d. Hours spent - relaxing and social! 31 d. Hours spent - relaxing and socializing 

31 e. Hours spent - providing care 30e. Hours spent - providing care 31 e. Hours spent - providing care 

31 f. Hours spent - commuting to 30f. Hours spent - commuting to class 31 f. Hours spent - commuting to class 

33. Some students during their academic career have a 

34. Doubting experiences 

35. Confirming experiences 

ED your interest in studying engineering, (open] 

D your interest in studying engineering, (open) 



6. If you are thinking of going to c 6. If you are thinking of going to gradu; 6. If you are thinking of going to graduate schooi in a fie 

1 8. What portion of the courses ) 1 7. What portion of the courses you he 1 8. What portion of the courses you have taken during 1 

1 9a. During the current schooi ye 1 8a. During the current schooi year, w 1 9a. During the current schooi year, what portion of you 

19d. Teaching methods - Labs 18d. Teaching methods - Labs 19d. Teaching methods - Labs 

19e. Teaching methods - Semine 18e. Teaching methods - Seminars 19e. Teaching methods - Seminars 

8b. I am a competitive person 8b. I am a competitive person 

8e. I strive to get higher grades than my ciassmates 8e. I strive to get higher grades than my ciassmates 

8f. The educationai institution I am attending promotes competitive work 8f. The educationai institution I am attending promotes ( 

8g. My instructors often remind students that they need to do better thar 8g. My instructors often remind students that they need 

lyseif 

8h. I have easy access to work spaces where I can participate in peer s' 8h. I have easy access to work spaces where I can part 
8i. I am encouraged by my instructors to initiate or participate in peer sti 8i. I am encouraged by my instructors to initiate or partic 
8k. The educationai institution I am attending promotes coiiaborative wo 8k. The educationai institution I am attending promotes 
n, how often you participated in a peer study group 

13d. Satisfaction with tutoring 12d. Satisfaction with tutoring 13d. Satisfaction with tutoring 

13e. Satisfaction with academic < 12e. Satisfaction with academic advisii 13e. Satisfaction with academic advising 

14. Since January, how often ha\ 13. Since January, how often have yoi 14. Since January, how often have you taken courses w 

20. To what extent have your coi 1 9. To what extent have your courses 20. To what extent have your courses required your enc 

kiiis 

44.-49. Who am I questions 45.-50. Who am I questions 

1 0f. Confidence: Computer skiiis 9f. Confidence: Computer skiiis 
1 1f. Perceived importance: Comf lOf. Perceived importance: Computer skiiis 


thinking skiiis 



38.-41 . UPri questions 
vork 


urses 

sful 

experience academic difficuities 
experience personai difficuities 


^vantage to share them 


s 

dents 


/ays 

ademic activities 



ronment 

oup 

it 


rk 


in instructor 

3ting an assignment due the next day 
larticuiariy chaiienging or compiex 
Dnsiderabiy more time to turn in "A" quaiity worh 
: student project 

se over a passing grade in a math, science, engineering course 
leering course over a passing grade in a generai education course 

i/ith other students 
time for recreation 
;rience 

th the supervision of a professor 
work 

writing, math) 



side the classroom? (open) 

friends at school for emotional support? (open) 

friends at school? If you do, under what circumstances? (open) 

ipen) 


te with an engineering degree? 



SURV 7: C1Y4A Spring 07 

2. Do you intend to complete a major in engineering? 

3. What do you intend to major in? 

4. If you intend to double major, what is the second major you intend to 
complete? 


5. Do you intend to practice, conduct research in, or teach engineering 
for at least 3 years after graduation? 


49. Post-graduation plans (open) 


7b. Engineers make more money than most other professionals 
7e. Engineers are well paid 

7g. An engineering degree will guarantee me a job when I graduate 
ler people 

7c. My parents would disapprove if I chose a major other than 
engineering 


APPLESl 


7. Do you intend to complete a major in engineering? 

6. What is/are your current major(s) or intended major(s)? (Mark 
all that apply) 


48. Double check: responses for “Not applicable - I am not 
continuing in an engineering major” 


34. Do you see yourself pursuing a career in engineering? 

33. How sure are you about your plans after graduation? 

32a. What do you see yourself doing in the first 3 years after 
graduation: Job working in engineering profession 

32b. What do you see yourself doing in the first 3 years after 
graduation: Job working in non-engineering profession 

32c. What do you see yourself doing in the first 3 years after 
graduation: Go to graduate school in engineering (Masters/PhD) 

32d. What do you see yourself doing in the first 3 years after 
graduation: Go to professional school (business, law, medicine, 

etc.) 

32e. What do you see yourself doing in the first 3 years after 
graduation: Go to graduate school NOT in engineering 

(Masters/PhD) 

32f. What do you see yourself doing in the first 3 years after 
graduation: Other 

8b. Engineers make more money than most other professionals 
8e. Engineers are well paid 

8g. An engineering degree will guarantee me a job when I 
graduate 

8c. My parents would disapprove if I chose a major other than 
engineering 







7f. My parents want me to be an engineer 

7a. Technoiogy piays an important roie in soiving society's probiems 
7d. Engineers have contributed greatiy to fixing probiems in the worid 

8f. My parents want me to be an engineer 

8a. Technology plays an important role in solving society's 
problems 

8d. Engineers have contributed greatly to fixing problems in the 
world 

Not asked 

Not asked 

7h. A facuity member, academic advsior, teaching assistant or other 

8h. A faculty member, academic advsior, teaching assistant or 

university affiiiated person has encouraged and/or inspired me to study other university affiiiated person has encouraged and/or inspired 

engineering 

me to study engineering 

7i. A non-university affiliated mentor has encouraged and/or inspired 

8i. A non-university affiliated mentor has encouraged and/or 

me to study engineering. 

inspired me to study engineering. 

9d. Math ability 

lOd. Math ability 

9e. Science ability 

10e. Science ability 

9g. Ability to apply math & science principles in solving real world 

10g. Ability to apply math & science principles in solving real 

problems 

world problems 

9a. Self confidence (social) 

10a. Self confidence (social) 

9b. Leadership ability 

10b. Leadership ability 

9c. Public speaking ability 

10c. Public speaking ability 

9f. Communication skills 

lOf. Communication skills 

9h. Business ability 

lOh. Business ability 

9i. Ability to perform in teams 

101. Ability to perform in teams 


8a. Creative thinking is one of my strengths 

8c. I am skiiied at soiving probiems that can have muitipie soiutions 


9a. Creative thinking is one of my strengths (4 pt scaie) 
9b. I am skiiied at soiving probiems that can have muitipie 
soiutions (4 pt scaie) 




9j. Critical thinking skills 


10d. Math ability 
10e. Science ability 

lOg. Ability to apply math & science principles in solving real world 
problems 

10a. Self confidence (social) 

10b. Leadership ability 
10c. Public speaking ability 
lOf. Communication skills 
lOh. Business ability 
1 0i. Ability to perform in teams 

8b. I am familiar with what a practicing engineer does 

25. How much exposure have you had to a professional engineering 
environment as a visitor, intern, or employee 

j engineers? 
igineers? 


lOj. Critical thinking skills (5 pt scale) 


lid. Math ability 

lie. Science ability 

llg. Ability to apply math & science principles in solving real 
world problems 

lla. Self confidence (social) 

llb. Leadership ability 

1 1 c. Public speaking ability 
Ilf. Communication skills 

l lh. Business ability 

1 1 i. Ability to perform in teams 


How did you gain your knowledge about the engineering 
profession? 

26a. From being a visitor 
26b. From being a co-op student 
26c. From being an intern 
26d. From being an employee 
26e. From a family member 
26f. From a close friend 
26g. From other 


27. Yes/No: Do any of your immediate family members (parents, 
siblings) hold an engineering degree? 

24. Before college, how much knowledge did you have about 
the engineering profession? 

25. Since entering college, how much knowledge have you 
gained about the engineering profession? 



17b. Teaching methods - Individual projects 


17c. Teaching methods - Team projects 

Deleted 

Deleted 

Deleted 

Deleted 


18. Importance of non-engineering activities 

20. Importance of non-engineering activities 

19. Frequency of involvement in non-engineering activities 

21. Frequency of involvement in non-engineering activities 

22. Level of involvement in student engineering activities such 
as engineering societies 

23. Since coming to college, have you had any research 

31 . Research experiences since coming to college 
20a. Pressure - course load 
20b. Pressure - course pace 

experiences? 

18a. Pressure - course load 
18b. Pressure - course pace 

20c. Pressure - Balance between social and academic life 

18c. Pressure - Balance between social and academic life 

21 . How well are you meeting workload demands of your coursework? 

1 6. How well are you meeting workload demands of your 
coursework? 


22. How stressed do you feel in your coursework right now? 17. How stressed do you feel in your coursework right now? 

23. Do you have any concerns about your ability to finance your college 28. Do you have any concerns about your ability to finance your 

education? college education? 


14a. Came late to liberal arts class 
14b. Skipped liberal arts class 

14c. Turned in liberal arts assignments that did not reflect your best 
work 

14d. Turned in liberal arts assignments late 

15a. Came late to liberal arts class 
15b. Skipped liberal arts class 

15c. Turned in liberal arts assignments that did not reflect your 
best work 

15d. Turned in liberal arts assignments late 

13a. Came late to engineering class 
13b. Skipped engineering class 

13c. Turned in engineering assignments that did not reflect your best 
work 

13d. Turned in engineering assignments late 

14a. Came late to engineering class 
14b. Skipped engineering class 

14c. Turned in engineering assignments that did not reflect your 
best work 

14d. Turned in engineering assignments late 





Constructs 1 1 a + 1 1b 


15a. Faculty during class 

15b. Faculty during office hours 

1 5c. Faculty outside of class or office hours 

15d. TAs during class 

15e. TAs during office hours 

1 5f. TAs outside of class or office hours 

Constructs 12a + 12b 


1 1 a. Quality of instruction by faculty 
1 1 b. Quality of advising by faculty 
1 1 c. Availability of faculty 
lid. Quality of instruction by TAs 
1 1 e. Quality of advising by TAs 
Ilf. Availability of TAs 

Constructs 13a + 13b 


12a. Computer facilities 
12b. Libraries 
12c. Classrooms 
12d. Laboratories 


27. Qverall quality of collegiate experience 


Constructs 1 la + 11b 


19a. Instructors during class 

19b. Instructors during office hours 

1 9c. Instructors outside of class or office hours 


12a. Quality of instruction by instructors 
12b. Quality of advising by instructors 
12c. Availability of instructors 
12d: Satisfaction: Academic advising 


13. Qverall quality of collegiate experience 



1 . What is your expected year of graduation from coiiege? 

32. What is your expected GPA this academic term 

33. Sex 

34. Age on 12/31 this year 

35. Ethnic background 

36. Maritai status 

37. Number of dependents 

38. Citizenship 


39. Born in U.S. 

40. Did one or more of your parents/guardians immigrate? 

41 . Is Engiish your first ianguage 

24a. How weii do you meet your coiiege expenses - seif (income) 
24b. Coiiege expenses - seif (savings) 

24c. Coiiege expenses - parents and famiiy 
24d. Coiiege expense - empioyer support 
24e. Coiiege expenses - schoiarships and grants 
24f. Coiiege expenses - ioans 


4. What year do you expect to compiete your undergraduate 
degree? 

30. What is your expected GPA this academic term 

31 . What is your expected GPA overaii 

35. Sex 

36. Raciai or ethnic identification 


37. Citizenship status 

2. What is your current academic standing? (freshman, 
sophomore, junior, senior, 5*^ year senior, graduate student, 
other) 

3. When you entered this institution, were you: (first-time, 
returning, transfer student) 

5. What were you most interested in majoring in when you first 
came to university? 

42. Fuii-time/part-time student 


38. Born in U.S. 

39. Did one or more of your parents/guardians immigrate? 

40. Is English your first language 

41 . Are you a first-generation college student? 



42. Highest ievei of education of mother 

44. Highest ievei of education of mother 

43. Highest ievei education of father 

45. Highest ievei education of father 

44. Best estimate of parents' totai income 

45. Description of famiiy 

43. Description of famiiy 


48b. Did you participate in engineering-reiated research iast summer? 
ETD18a. During your undergraduate years, how many months of 
experience with part-time academic research work on campus 
ETD18b. During your undergraduate years, how many months of 
experience with fuii-time academic research work on campus 
48a. Did you participate in an engineering reiated internship/job iast 
summer? 

48c. Did you participate in engineering-reiated coursework iast 
summer? 

46. What did you do iast summer (2006) that was particuiariy important 
to you? (open) 

47. Did your experience advance your interest in studying engineering? 
(open) 

ETD18C. During your undergraduate years, how many months of 
experience with part-time engineering work (internship, co-op, summer 
job) 

ETD18d. During your undergraduate years, how many months of 
experience with fuii-time engineering work (internship, co-op, summer 
job) 

ETD19. Short description of your experience(s) with academic research 
and/or professionai engineering (open) 


26a. Hours spent - preparing for ciass 

26b. Hours spent - working for pay 

26c. Hours spent - participating in co-curricuiar activities 

26d. Hours spent - reiaxing and sociaiizing 


29a. Hours spent - preparing for ciass 

29b. Hours spent - working for pay 

29c. Hours spent - participating in co-curricuiar activities 

29d. Hours spent - reiaxing and sociaiizing 


26e. Hours spent - providing care 29e. Hours spent - providing care 

26f. Hours spent - commuting to ciass 29f. Hours spent - commuting to ciass 


28. I started at this institution... 47. I started at this institution... 

29. Decision to continue engineering - reasons 48. Decisions to continue engineering - reasons 


30. Doubts - reasons 


49. Doubts - reasons 






6. If you are thinking of going to graduate schooi in a fieid OTHER 
THAN engineering, piease mark your most probabie area of study. 

16. What portion of the courses you have taken during the current 
schooi year have been taught primariiy by graduate students? 

17a. During the current schooi year, what portion of your ciasses have 
used the foiiowing teaching methods - Lectures 
17d. Teaching methods - Labs 
17e. Teaching methods - Seminars 


competitive work 

to do better than other students to obtain high grades 


icipate in peer study/discussion sessions with my feiiow students 
cipate in peer study sessions with my feiiow students 
coiiaborative work 


fhich required your enagement in individuai or group projects 
jagement in individuai and/or group projects 


1 . What schooi are you currentiy attending? 

51 . How did you iearn about the survey 

52. Wouid you have been wiiiing to take the survey if the 
compensation was... 

50. Is there anything you want to teii us about your experiences 
in engineering that we haven’t aiready asked you about? 





APPLES2 


7. Do you intend to complete a major in engineering? 

5. What is your current major or first choice of major 
(Mark one) 

6. What is your second choice of major or second 
major/minor (if applicable)? 

31 . Do you see yourself continuing in an engineering 
major? 

8. Do you intend to practice, conduct research in, or 
teach engineering for at least 3 years after graduation? 

32. Do you see yourself pursuing a career in 
engineering? 


33a. How likely is it that you would do each of the 
following after graduation?: Work in an engineering job 
33b. How likely is it that you would do each of the 
following after graduation?: Work in a non-engineering 
job 

33c. How likely is it that you would do each of the 
following after graduation?: Go to graduate school in an 
engineering discipline 


33d. How likely is it that you would do each of the 
following after graduation?: Go to graduate school in a 
non-engineering discipline 


9b. Engineers make more money than most other 
professionals 

9e. Engineers are well paid 

9g. An engineering degree will guarantee me a job when 
I graduate 


9c. My parents would disapprove if I chose a major other 
than engineering 







9f. My parents want me to be an engineer 


9a. Technology plays an important role in solving 
society's problems 

9d. Engineers have contributed greatly to fixing 
problems in the world 

9n. Engineering skills can be used for the good of 
society 


Not asked 


9h. A faculty member, academic advsior, teaching 
assistant or other university affiliated person has 
encouraged and/or inspired me to study engineering 

9i. A non-university affiliated mentor has encouraged 
and/or inspired me to study engineering. 

9j. A mentor has introduced me to people and 
opportunities in engineering 
10c. Agree/disagree: A mentor has supported my 
decision to major in engineering. 

lid. Math ability 

lie. Science ability 

l lg. Ability to apply math & science principles in solving 
real world problems 

1 1 a. Self confidence (social) 

llb. Leadership ability 

llc. Public speaking ability 
Ilf. Communication skills 

llh. Business ability 

1 1 i. Ability to perform in teams 


10a. Creative thinking is one of my strengths (4 pt scale) 
10b. I am skilled at solving problems that can have 
multiple solutions (4 pt scale) 






1 1j. Critical thinking skills (5 pt scale) 


12d. Math ability 
12e. Science ability 

12g. Ability to apply math & science principles in solving 
real world problems 

12a. Self confidence (social) 

12b. Leadership ability 
12c. Public speaking ability 
12f. Communication skills 
12h. Business ability 
121. Ability to perform in teams 


28. How much exposure have you had to a professional 
engineering environment as a visitor, intern, or 
employee 

How did you gain your knowledge about the engineering 
profession? 

29a. From being a visitor 

29b. From being a co-op student or intern 

29c. From being an employee 
29d. From a family member 
29e From a close friend 

29g. From other 


29f. From school-related experiences (professor, class) 
30. Yes/No: Do any of your immediate family members 
(parents, siblings) hold an engineering degree? 

26. Before college, how much knowledge did you have 
about the engineering profession? 

27. Since entering college, how much knowledge have 
you gained about the engineering profession? 




14a. Teaching methods - individual projects 
14b. Teaching methods - team projects 


22. Importance of non-engineering activities 

23. Frequency of involvement in non-engineering 
activities 

24. Level of involvement in student engineering activities 
such as engineering societies 

25. Since coming to college, have you had any research 
experiences? 

20a. Pressure - course load 
20b. Pressure - course pace 

20c. Pressure - Balance between social and academic 
life 

1 8. How well are you meeting workload demands of your 
coursework? 

19. How stressed do you feel in your coursework right 
now? 

34. Do you have any concerns about your ability to 
finance your college education? 


17a. Came late to liberal arts class 

17b. Skipped liberal arts class 

17c. Turned in liberal arts assignments that did not 

reflect your best work 

17d. Turned in liberal arts assignments late 


16a. Came late to engineering class 

16b. Skipped engineering class 

16c. Turned in engineering assignments that did not 

reflect your best work 

16d. Turned in engineering assignments late 






21a. Instructors during class 

21b. Instructors during office hours 

21c. Instructors outside of class or office hours 


13a. Quality of instruction by instructors 
13b. Quality of advising by instructors 
13c. Availability of instructors 
13d: Satisfaction: Academic advising 


15. Qverall quality of collegiate experience 


9k. I feel good when I am doing engineering 
9m. I think engineering is fun 

90. I think engineering is interesting 

91. I like to build stuff 

9p. I like to figure out how things work 



35. What is your cumulative GPA? 

36. Sex 

38. How old are you? 

37. Racial or ethnic identification 


39. Citizenship status 

2. What is your current academic standing? (freshman, 
sophomore, junior, senior, 5*^ year senior, graduate 
student, other) 

3. When you entered this institution, were you: (first- 
time, returning, transfer student) 

4. What were you most interested in majoring in when 
you first came to university? 

44. Full-time/part-time student 


45. Which of the following best describes where you are 
living now while attending college? 


40. Born in U.S. 

41 . Did one or more of your parents/guardians 
immigrate? 

42. Is English your first language 

43. Are you a first-generation college student? 





47. Highest level of education of mother 

48. Highest level education of father 


46. Description of family 




1 . What school are you currently attending? 


50. Is there anything you want to tell us about your 
experiences In engineering that we haven’t already 
asked you about? 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 


Selected Variables and Items from PIE to APPLESl to APPLES2 


OFFICIAL VARIABLES/CONSTRUCTS 
1a: Academic Persistence 
1b: Professionai Persistence 
2a: Motivation (Financiai) 

2b: Motivation (Famiiy Infiuence) 

2c: Motivation (Sociai Good) 

2d: Motivation (High Schooi Mentor infiuence) 

2e: Motivation (Mentor Infiuence) 

3a: Confidence in Math & Science Skiiis 
3b: Confidence in Professionai and Interpersonai Skiiis 
3c: Confidence in Soiving Open-ended Probiems 
4a: Perceived importance in Math & Science Skiiis 

4b: Perceived Importance in Professional & Interpersonal Skills 
5: Knowledge of the Engineering Profession 

5a. Knowledge of the Engineering Profession (Change over time) 
5b. Knowledge of the Engineering Profession (Sources) 

6a: Exposure to Project-Based Learning Methods: Individual 
6b: Exposure to Project-Based Learning Methods: Team 
7: Collaborative Work Style 
8: Extracurricular Fulfillment (Non-engineering) 

8b: Extracurricular Involvement (Engineering) 

8c: Research Experience 
9: Curriculum Overload 
10: Financial Difficulties 

11a: Academic Disengagement - Liberal Arts courses 

11b: Academic Disengagement - Engineering-Related courses 

11c: Academic Disengagement - Overall 

12: Frequency of Interaction with Instructors 

13a: Satisfaction with Instructors 

13d. Satisfaction with academic advising 

13b: Satisfaction with Academic Facilities 

13c: Overall Satisfaction with Collegiate Experience 

Intrinsic Motivation (Psychological) 

Intrinsic Motivation (Behavioral) 

DEMOGRAPHIC VARIABLES 

Survey verification: What school are you currently attending? 

Expected Graduation Date 
Expected GPA academic term 
Expected GPA overall 
Sex 
Age 

Ethnicity 
Marital status 
Dependents 


SURV-1: 
CIYIA 
Winter 04 

Y 

Not Asked 
Different 
Different 
Different 
Not Asked 
Not Asked 
Different 
Different 
Different 
Different 

Different 
Not Asked 

Not Asked 
Not Asked 

Y 

Y 

Different 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 

Y 

Not Complete 
Different 
Different 

Y 

Different 
Not Asked 
Not Asked 
Not Asked 


Not Asked 

Y 

Not Asked 
Not Asked 

Y 

Y 

Y 

Y 

Y 


Citzenship Y 

Current academic standing Not Asked 

Traditionai/nontraditionai student Not Asked 

Preiiminary area of interest Not Asked 

Fuii time/part-time student Not Asked 

Year graduated from high schooi Y 

High schooi community Y 

Average grade in high schooi Y 

Residence in coiiege Y 

Roommates in coiiege Y 

Disabiiities Y 

Assistive technoiogies Y 

Born in U.S. Not Asked 

Parents/guardians immigrate? Not Asked 

Engiish first ianguage Not Asked 

First Generation Coiiege Student Not Asked 

How weii do you meet your coiiege expenses - seif (income) Y 

Coiiege expenses - seif (savings) Y 

Coiiege expenses - parents and famiiy Y 

Coiiege expense - empioyer support Y 

Coiiege expenses - schoiarships and grants Y 

Coiiege expenses - ioans Y 

Coiiege expenses - other sources Y 

Highest ievei of education of mother Y 

Highest ievei education of father Y 

Best estimate parents' income Y 

Description of Famiiy Income Not Asked 


UNOFFiCIAL VARiABLES 

Research: Did you participate in engineering-reiated research iast 
summer? 

Research: ETD18a. During your undergraduate years, how many 
months of experience with part-time academic research work on 
campus 

Research: ETD18b. During your undergraduate years, how many 
months of experience with fuii-time academic research work on 
campus 

Research: Did you participate in engineering-reiated coursework iast 
summer? 

Extra-curricuiar: Did your experience advance your interest in 
studying engineering? (open) 

Extra-curricular: What did you do iast summer that was particuiariy 
important to you? (open) 

Extra-curricular: Short description of your experience(s) with 
academic research and/or professionai engineering (open) 
Engineering Extra-curricuiar: Some students participate in design 
competitions, internships, and ciubs. In the space provided identify 
engineering-reiated activities you have participated in outside of 
ciass.(open) 

Engineering Profession: During your undergraduate years, how 
many months of experience with part-time engineering work 
(internship, co-op, summer job) 


Not Asked 

Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 

Y 

Not Asked 


Engineering Profession: During your undergraduate years, how 
many months of experience with fuii-time engineering work (internship, 
co-op, summer job) Not Asked 

Engineering Profession: Did you participate in an engineering reiated 
internship/job iast summer? Not Asked 

NSSE: Hours spent - preparing for ciass Y 

NSSE: Hours spent - working for pay Y 

NSSE: Hours spent - participating in co-curricuiar activities Y 

NSSE: Hours spent - reiaxing and sociaiizing Y 

NSSE: Hours spent - providing care Y 

NSSE: Hours spent - commuting to ciass Y 

Confirm/Doubt: I started at this institution . . . Not Asked 

Confirm/Doubt: Decision to continue engineering - reasons Not Asked 

Confirm/Doubt: Doubts - reasons Not Asked 

Confirm/Doubt; Identify any ciasses (in high schooi or coiiege so far) 
this year that have STRONGLY REINFORCED your interest in 
studying engineering, (open) Y 

Confirm/Doubt; Identify any ciasses (in high schooi or coiiege so far) 
this year that have STRONGLY WEAKENED your interest in studying 
engineering, (open) Y 


MISCELLANEOUS ITEMS NOT ASSIGNED TO A CONSTRUCT OR 
DELETED OVER TIME 

Open-ended question: Is there anything you want to teii us about your 


experiences in engineering that we haven’t aiready asked you about? Not Asked 

Grad School: If you are thinking of going to graduate schooi in a fieid 

OTHER THAN engineering, piease mark your most probabie area of 

study. Y 

Course taught by grad students: What portion of the courses you 

have taken during the current schooi year have been taught primariiy 

by graduate students? Y 

Teaching methods - lectures Y 

Teaching methods - labs Y 

Teaching methods - seminars Y 

Competitive work style (personal): I am a competitive person Not Asked 

Competitive work style (personal): I strive to get higher grades than 

my ciassmates Not Asked 

Competitive work style (institutional): The educationai institution I 

am attending promotes competitive work Not Asked 

Competitive work style (institutional): My instructors often remind 

students that they need to do better than other students to obtain high 

grades Y 

Competitive work style (institutional): I prefer keeping good ideas to 

myseif Not Asked 

Competitive work style (institutional): My instructors grade on a 

curve Y 

Collaborative work style (institutional): I have easy access to work 

spaces where I can participate in peer study/discussion sessions with 

my feiiow students Y 


Collaborative work style (institutional): I am encouraged by my 

instructors to initiate or participate in peer study sessions with my 

feiiow students Y 

Collaborative work style (institutional): The educationai institution I 

am attending promotes coiiaborative work Not Asked 

Collaborative work style (institutional): Since the beginning of faii 

term, how often you participated in a peer study group Not Asked 

Satisfaction with academic services: Satisfaction with tutoring Y 

Exposure to PEL Methods: Since this term, how often have you 

taken courses which required your enagement in individuai or group 

projects Not Asked 

Exposure to PEL Methods: To what extent have your courses 

required your engagement in individuai and/or group projects Not Asked 

Exposure to PEL Methods:: I have strong probiem soiving skiiis Y 

Who Am I questions Not Asked 

Confidence in computer/programming skills Y 

Perceived importance of computer skills Y 

Perceived importance of critical thinking skills Not Asked 

UPri questions - Motivation, Self-Esteem, Ethnic identity Not Asked 

Motivation (Eelief That Engineers Improve Welfare of Society 

Through Creative Work): I enjoy figuring out how things work Y 

Motivation (Eelief That Engineers Improve Welfare of Society 

Through Creative Work): Engineers are innovative Y 

Motivation (Eelief That Engineers Improve Welfare of Society 

Through Creative Work): Engineers are creative Y 

I enjoy the subjects of science and math the most Y 

Creative thinking is one of my strengths Y 

Peers: Studying in a group is better than studying by myseif Y 

I enjoy taking iiberai arts courses more than science and math courses Y 

Peers: My friends are supportive of me when I am academicaiiy 
successfui Not Asked 

Peers: I can count on my friends at schooi for emotionai support when 
I experience academic difficuities Not Asked 

Peers: I can count on my friends at schooi for emotionai support when 
I experience personai difficuities Not Asked 

Importance: Getting higher grades than my ciassmates Y 

Importance: Influencing social values Y 

Importance: Becoming an authority in my field Y 

Importance: Keeping goood ideas to myself unless it is to my 
advantage to share them Y 

Importance: Helping to promote racial understanding Y 

Importance: Becoming a community leader Y 

Importance: Helping others who are in difficulty Y 

Importance: When playing any game, playing to win Y 

Importance: Developing a meaningful philosophy of life Y 

Importance: Becoming a practicing engineer Y 

Importance: Getting along with others Y 

Importance: Working as part of a team Y 

Importance: Becoming a student government official Y 

Importance: Establishing relationships with engineering students Y 


Importance: Establishing relationships with non-engineering students Y 

Confidence: Analytical and problem solving skills Y 

Confidence: General knowledge Y 

Confidence: Knowledge of a particular field or discipline Y 

Confidence: Interest in studying engineering Y 

Satisfaction: Opportunities for community service Y 

Frequency: Felt that your courses inspired you to think in new ways Y 

Frequency: Felt you did not have enough time to pursue nonacademic 
activities Y 

Frequency: Worried about keeping up with your schoolwork Y 

Frequency: Felt you did not have a "social life" Y 

Frequency: Felt stressed Y 

Peers: Frequency: Participated in a peer study group Y 

Frequency: Visited or worked in a commercial engineering 

environment Not Asked 


Frequency: Socialized with someone of another racial/ethnic group 
Frequency: Discussed racial issues 

Frequency: Attended a racial/cultural awareness workshop/event 
Came late to math class 
Skipped math class 

Turned in math assignments that did not reflect your best work 

Turned in math assignments late 

Thought math classes were boring 

Came late to science class 

Skipped science class 

Turned in science assignments that did not reflect your best work 
Turned in science assignments late 
Thought science classes were boring 

Frequency: Asked for advice about managing your coursework 
Frequency: Worked with people who have diverse backgrounds 
Frequency: Spent time in a non-academic role 
Frequency: Requested feedback on course assignments from an 
instructor 

Frequency: "Crammed" all night studying for an exam or completing an 
assignment due the next day 

Frequency: Worked collaboratively on an assignment that was 
particularly challenging or complex 

Frequency: Decided to turn in "C" quality work over spending 
considerably more time to turn in "A" quality work 
Frequency: Had a research experience on a faculty or graduate 
student project 

Frequency: Prioritized a good grade in a general education course over 
a passing grade in a math, science, engineering course 
Frequency: Prioritized a good grade in a math, science, or engineering 
course over a passing grade in a general education course 
Frequency: Studied regularly in blocks of 2 hours or more 
Frequency: Took a seminar course to discuss and argue ideas with 
other students 

Frequency: Reduced time spent on course work to have more time for 
recreation 


Not Asked 
Not Asked 
Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 
Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 


Frequency: Asked for advice about managing your coiiege experience Not Asked 
Frequency: Created a project outside of your academic work with the 
supervision of a professor Not Asked 

Frequency: Monitored how you spent your time on your course work Not Asked 
Frequency: Asked for heip to strengthen a particuiar skiii (e.g., writing, 
math) Not Asked 

Frequency: Reduced your course ioad to improve your grades Not Asked 

Confidence: Seif-confidence (inteiiectuai) Y 

Frequency: Worked on ciass projects Y 

Frequency: Fleid a study group Y 

Frequency: Took a specific iecture-type ciass Y 

Frequency: Took a specific iaboratory ciass Y 

Frequency: Worked on homework Y 

Frequency: Reviewed ciass materiai Y 

Frequency: Prepared for ciass exams Y 

Frequency: Wrote ciass reports Y 

In what ways do you interact with other engineering students outside 
the ciassroom? (open) Y 

When and under what circumstances (if any) do you reiy on you friends 
at schooi for emotionai support? (open) Not Asked 

Do you feei uncomfortabie seeking emotionai support from your friends 
at schooi? If you do, under what circumstances? (open) Not Asked 

What is the race/ethnicity of your six ciosest friends at schooi? (open) Not Asked 

43-46. Phii Beii's technoiogy questions: How often do you use the 

foiiowing technoiogies in your personai iife outside of schooi/schooi- 

reiated activities? Y 

What is the first word or phrase you think of to describe your favorite 

professor? Y 

Do you beiieve your peers wouid agree with this description? Y 

What inteiiectuai, personai, financiai, and other chaiienges do you feei 

you may need to overcome ito graduate with an engineering degree? Y 


UPri QUESTIONS 

Group Identification Scaie: I identify with engineering students 
Group Identification Scaie: I am giad to beiong to a group of 
engineering students 

Group Identification Scaie: I feei heid back by engineering students 
Group Identification Scaie: I think engineering students work weii 
together 

Group Identification Scaie: I see myseif as an important part of 
engineering students on campus 

Group Identification Scaie: I fit in weii with the other engineering 
students 

Group Identification Scaie: I consider engineering students to not be 
important 

Group Identification Scaie: I feei uneasy with other engineering 
students 

Group Identification Scaie: I feei strong ties to engineering students 
SIMS: Intrinsic Motivation: I think engineering is interesting 
SIMS: Intrinsic Motivation: I think engineering is pieasant 


Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 


SIMS: Intrinsic Motivation: Majoring in engineering is fun Not Asked 

SIMS: Intrinsic Motivation: I feei good when I am doing engineering 
activities Not Asked 

SIMS: Identified Reguiation: I am majoring in engineering for my own 
good Not Asked 

SIMS: Identified Reguiation: I think engineering is good for me Not Asked 

SIMS: Identified Reguiation: It is my personai decision Not Asked 

SIMS: Identified Reguiation: I beiieve engineering is important for me Not Asked 

SIMS: Amotivation: There may be good reasons to major in 

engineering, but personaiiy, I don’t see any Not Asked 

SIMS: Amotivation: I am majoring in (considering majoring in) 

engineering, but I am not sure if it is worth it Not Asked 


SIMS: Amotivation: I don’t know. I don’t see what the activity brings me 
SIMS: Amotivation: I am doing it, but am not sure it is a good thing to 
pursue 

SIMS: External Regulation: I am supposed to major in engineering 
SIMS: External Regulation: Majoring in engineering is something that I 
have to do 

SIMS: External Regulation: I don’t have any choice 
SIMS: External Regulation: I feel that I have to do it 
Self-Esteem: On the whole, I am satisfied with myself 
Self-Esteem: At times, I think I am no good at all 
Self-Esteem: I feel that I have a number of good qualities 
Self-Esteem: I am able to do things as well as most other people 
Self-Esteem: I feel I do not have much to be proud of 
Self-Esteem: I certainly feel useless at times 
Self-Esteem: I feel that I’m a person of worth, at least on an equal 
plane with others 

Self-Esteem: I wish I could have more respect for myself 
Self-Esteem: All in all, I am inclined to feel that I am a failure 
Self-Esteem: I take a positive attitude toward myself 
MIBI Centrality Engineering: Overall, being an engineering student has 
very little to do with how I feel about myself 

MIBI Centrality Engineering: In general, being an engineering student 
is an important part of my self image 

MIBI Centrality Engineering: My destiny is tied to the destiny of other 
engineering students 

MIBI Centrality Engineering: Being an engineering student is 
unimportant to my sense of what kind of person I am 
MIBI Centrality Engineering: I have a strong sense of belonging to the 
engineering student community 

MIBI Centrality Engineering: I have a strong attachment to other 
engineering students 

MIBI Centrality Engineering: Being an engineering student is an 
important reflection of who I am 

MIBI Centrality Engineering: Being an engineering student is not a 

major factor in my social relationships 

MIBI Private Regard Engineering: I feel good about engineers 

MIBI Private Regard Engineering: I am happy that I am going to be an 

engineer 


Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 



MIBI Private Regard Engineering: I feei that engineers have made 
major accompiishments and advancements 
MIBI Private Regard Engineering: I often regret that I am going to 
become an engineer 

MIBI Private Regard Engineering: I am proud to be an engineer 
MIBI Private Regard Engineering: I feei that the engineering 
community has made vaiuabie contributions to this society 
MIBI Public Regard Engineering: Overall, engineers are considered 
good by others 

MIBI Public Regard Engineering: In general, others respect engineers 
MIBI Public Regard Engineering: Most people consider engineers, on 
the average, to be more ineffective than other professionals 
MIBI Public Regard Engineering: Engineers are not respected by the 
broader society 

MIBI Public Regard Engineering: In general, other professionals view 
engineers in a positive manner 

MIBI Public Regard Engineering: Society views engineers as an 
asset+B283 

MIBI Centrality Ethnicity: Overall, being a member of my ethnic group 

has very little to do with how I feel about myself 

MIBI Centrality Ethnicity: In general, being a member of my ethnic 

group is an important part of my self image 

MIBI Centrality Ethnicity: My destiny is tied to the destiny of other 

members of my ethnic group 

MIBI Centrality Ethnicity: Being a member of my ethnic group is 
unimportant to my sense of what kind of person I am 
MIBI Centrality Ethnicity: I have a strong sense of belonging to my 
ethnic group community 

MIBI Centrality Ethnicity: I have a strong attachment to other members 
of my ethnic group 

MIBI Centrality Ethnicity: Being a member of my ethnic group is an 
important reflection of who I am 

MIBI Centrality Ethnicity: Being a member of my ethnic group is not a 
major factor in my social relationships 

MIBI Centrality Gender: Overall, being a member of my gender has 
very little to do with how I feel about myself 

MIBI Centrality Gender: In general, being a member of my gender is an 
important part of my self-image 

MIBI Centrality Gender: My destiny is tied to the destiny of other 
members of my gender 

MIBI Centrality Gender: Being a member of my gender is unimportant 
to my sense of what kind of person I am 
MIBI Centrality Gender: I have a strong sense of belonging to my 
gender community 

MIBI Centrality Gender: I have a strong attachment to other members 
of my gender 

MIBI Centrality Gender: Being a member of my gender is an important 
reflection of who I am 

MIBI Centrality Gender: Being a member of my gender is not a major 
factor in my social relationships 


Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


KEY 


Different 



SURV 2: 

SURV 3: 

SURV 4: 

SURV 5: 

SURV 6: 

SURV 7: 

CIYIB Spring 

C1Y2A 

C1Y2B Spring 

C1Y3A 

C1Y3B Spring 

C1Y4A Spring 

04 

Fall 04 

05 

Fall 05 

06 

07 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Different 

Different 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Different 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Different 

Y 

Y 

Y 

Y 

Y 

Different 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 



Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 


Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 


Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 



Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Different 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Different 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Different 

Y 


Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Y 

Not Asked 

Y 

Y 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Not Asked 

Y 

Not Asked 



Y 

Y 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Not Asked 


Not Asked 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 


Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 



Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 



Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 



Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 



Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 



Items comprising this variabie were iater modified 



APPLESl 
Spring 07 


APPLES2 
Winter 08 


Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Not Asked 
Not Asked 
Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Not Asked 
Not Asked 


Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Not Asked 
Not Asked 


Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Y 

Y 

Y 


Y 

Not Asked 
Not Asked 

Y 

Y 

Y 

Y 

Not Asked 
Not Asked 



Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Y 

Y 

Y 

Y 

Not Asked 

Not Asked 

Y 

Y 


Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 

Not Asked 

Not Asked 

Not Asked 



Not Asked 


Not Asked 


Not Asked 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 

Y 

Not Asked 


Not Asked 

Not Asked 

Not Asked 

Not Asked 


Y 

Y 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 


Not Asked 



Not Asked 


Not Asked 


Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 


Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 


Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 

Not Asked Not Asked 



Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 



Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 


Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 



Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 
Not Asked 
Not Asked 
Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 
Not Asked 

Not Asked 
Not Asked 

Not Asked 

Not Asked 



Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 

Not Asked 




Cronbach's Alphas for Multi-item Scales 

SURV-1: 

SURV 2: 

SURV 3: 



CIYIA 

CIYIB Spring 

C1Y2A 



Winter 04 

04 Frosh 

Fall 04 



Frosh 


Soph 


OFFICIAL VARIABLES/CONSTRUCTS 




1 

1a: Academic Persistence 

Single Item 

Single Item 

Single Item 

2 

1b: Professionai Persistence 

Single Item 

Single Item 

Single Item 

3 

2a: Motivation (Financiai) 

Different 

Y 

Y 

4 

2b: Motivation (Famiiy Infiuence) 

Different 

Y 

Y 

5 

2c: Motivation (Sociai Good) 

Different 

Y 

Y 

6 

2d: Motivation (High Schooi Mentor Infiuence) 

— 

Y 

— 

7 

2e: Motivation (Mentor Influence) 

— 

— 

Y 

8 

3a: Confidence in Math & Science Skills 

Different 

Y 

Y 

9 

3b: Confidence in Professional and Interpersonal Skills 

Different 

Y 

Y 

10 

3c: Confidence in Solving Open-ended Problems 

Different 

Y 

Y 

11 

4a: Perceived Importance in Math & Science Skills 

Different 

Y 

Y 

12 

4b: Perceived Importance in Professional & Interpersonal Skills 

Different 

Different 

Different 

13 

5: Knowledge of the Engineering Profession 

--- 

Y 

Y 

14 

5a. Knowledge of the Engineering Profession (Change over time) 

— 

— 

— 

15 

5b. Knowledge of the Engineering Profession (Sources) 

— 

— 

— 

16 

6a: Exposure to Project-Based Learning Methods: Individual 

Single Item 

Single Item 

Single Item 

17 

6b: Exposure to Project-Based Learning Methods: Team 

Single Item 

Single Item 

Single Item 

18 

7: Collaborative Work Style 

Different 

Different 

Y 

19 

8: Extracurricular Fulfillment (Non-engineering) 

___ 

___ 

Y 

20 

8b: Extracurricular Involvement (Engineering) 

— 

— 

— 

21 

8c: Research Experience 

— 

— 

— 

22 

9: Curriculum Overload 

— 

— 

Y 

23 

10: Financial Difficulties 

— 

— 

Single Item 

24 

11a: Academic Disengagement - Liberal Arts courses 

— 

Y 

Y 

25 

11b: Academic Disengagement - Engineering-Related courses 

Y 

Y 

Y 

26 

11c: Academic Disengagement - Overall 

Not Complete 

Y 

Y 

27 

12: Frequency of Interaction with Instructors 

Different 

Different 

Y 

28 

13a: Satisfaction with Instructors 

Different 

Different 

Y 

29 

13d. Satisfaction with academic advising 

Single Item 

Single Item 

Single Item 

30 

13b: Satisfaction with Academic Facilities 

Different 

Y 

Y 


31 

13c: Overall Satisfaction with Collegiate Experience 

— 

Single Item 

Single Item 

32 

Intrinsic Motivation (Psychological) 

— 

— 

— 

33 

Intrinsic Motivation (Behavioral) 

— 

— 

— 


KEY 

Not Asked 

N/A Alpha cannot be calculated bee 
Single Item Single Item variable; alpha can 


SURV 4: 

SURV 5: 

SURV 6: 

SURV 7: 

APPLESl 

APPLESl 

APPLESl 

APPLESl 

APPLESl 

C1Y2B Spring 

C1Y3A 

C1Y3B Spring 

C1Y4A Spring 

Spring 07 

Spring 07 

Spring 07 

Spring 07 

Spring 07 

05 Soph 

Fall 05 
Junior 

06 Junior 

07 Senior 

Overall 

Frosh 

Soph 

Junior 

Senior 










Single Item 

Single Item 

Single Item 

Single Item 

Single Item 





Single Item 

Single Item 

Single Item 

Single Item 

Single Item 





Y 

0.76 

Y 

Y 

0.82 





Y 

0.85 

Y 

Y 

0.87 





Y 

0.70 

Y 

Y 

0.64 





— 

... 

... 

... 

... 





Y 

0.65 

Y 

Y 

0.60 





Y 

0.83 

Y 

Y 

0.82 





Y 

0.84 

Y 

Y 

0.80 





Y 

0.69 

Y 

Y 

0.68 





Y 

0.79 

Y 

Y 

0.79 





Y 

0.79 

Y 

Y 

0.83 





Y 

Y 

Y 

Y 

— 





... 

... 

... 

... 

N/A 





... 

... 

... 

... 

N/A 





Single Item 

Single Item 

Single Item 

Single Item 

— 





Single Item 

Single Item 

Single Item 

Single Item 

... 





Y 

Y 

Y 

... 

... 





Y 

0.85 

Y 

Y 

0.82 





... 

... 

... 

... 

Single Item 





... 

... 

Single Item 

Single Item 

Single Item 





Y 

0.81 

Y 

Y 

0.78 





Single Item 

Single Item 

Single Item 

Single Item 

Single Item 





Y 

0.58 

Y 

Y 

0.88 





Y 

0.70 

Y 

Y 

0.86 





Y 

Y 

Y 

Y 

Y 





Y 

0.69 

Y 

Y 

0.74 





Y 

0.84 

Y 

Y 

0.72 





Single Item 

Single Item 

Single Item 

... 

Single Item 





Y 

Y 

Y 

Y 

... 






Single Item 

Single Item 

Single Item 

Single Item 

Single Item 





— 

— 

— 

— 

— 





— 

— 

— 

— 

— 






:ause it's not a scale 
not be calculated 









APPLES2 
Winter 08 
Overall 

APPLES2 
Winter 08 
Frosh 

APPLES2 
Winter 08 
Soph 

APPLES2 
Winter 08 
Junior 

APPLES2 
Winter 08 
Senior 






Single Item 





Single Item 





Y 





Y 





Y 





— 





Y 





Y 





Y 





Y 





Y 





Y 





Y 





N/A 





N/A 





Single Item 





Single Item 





— 





Y 





Single Item 





Single Item 





Y 





Single Item 





Y 





Y 





Y 





Y 





Y 





Single Item 





— 











Appendix 4-C 
APPLES1 Instrument 


© APS 2006, 2007 

ACADEMIC PATHWAYS OF PEOPLE LEARNING ENGINEERING SURVEY 

(APPLES) 

For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 

*1. What school are you currently attending? 

O [NAME OF INSTITUTION] 

O Other: I 

o 

I prefer not to answer 

* 2 . What is your current academic standing? 

o 

Freshman 

o 

Sophomore 
O Junior 
O Senior 

O Fifth year senior or more 

o 

Graduate student 

O other: I 

O 

I prefer not to answer 

*3. When you entered this institution were you: 

o 

A first-time college student 

o 

Returning or non-traditional college student 

o 

A transfer student from a two-year institution 

o 

A transfer student from a four-year institution 

o 

A transfer student from an institution that participates in a 3 -F 2 engineering program 
O I prefer not to answer 
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*4. What year do you expect to complete your undergraduate degree? 

O 2007 or earlier 

O 2008 

O 2009 

O 2010 
O 2011 

O 2012 or later 
O I prefer not to answer 

*5. What were you most interested in majoring in when you first came to university? 
(Choose one) 

o 

Arts and Humanities 

o 

Engineering 

o 

Math and Natural Sciences 
O Physical Sciences 
O Social Sciences 

O other: I 
O 

I prefer not to answer 

*6. What is/ are your current major(s) or intended major(s)? (Mark all that apply) 

□ Aerospace Engineering 

□ Chemical Engineering 

□ Civil Engineering 

□ Computer Science/Engineering 

□ Electrical Engineering 

□ Industrial Engineering 

□ Materials and Metallurgical Engineering 

□ Mechanical Engineering 

□ Arts and Humanities 

□ Math and Natural Sciences 

□ Physical Sciences 

□ Social Sciences 
O Other 

□ I prefer not to answer 


4C-2 
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*7. Do you intend to complete a major in engineering? 

O Definitely Not 
O Probably Not 
O Not Sure 
O Probably Yes 
O Definitely Yes 

o 

I prefer not to answer 


* 8 . We are interested in knowing why you are or were studying engineering. Please 
indicate below the extent to which the following reasons apply to you: 


I prefer 



Not a 
Reason 

Minimal 

Reason 

Moderate 

Reason 

Major 

Reason 

not to 
answer 

Technology plays an important role in 
solving society's problems 

o 

o 

o 

o 

O 

Engineers make more money than most 
other Drofessionals 

o 

o 

o 

o 

o 

My parent(s) would disapprove if 1 chose a 
major other than engineering 

o 

o 

o 

o 

o 

Engineers have contributed greatly to 
fixina Broblems in the world 

o 

o 

o 

o 

o 

Engineers are well paid 

o 

o 

o 

o 

o 

My parent(s) want me to be an engineer 

o 

o 

o 

o 

o 

An engineering degree will guarantee me a 
job when 1 graduate 

o 

o 

o 

o 

o 

A faculty member, academic advisor, 
teaching assistant or other university 
affiliated person has encouraged and/or 
inspired me to study engineering 

o 

o 

o 

o 

o 

A non-university affiliated mentor has 
encouraged and/or inspired me to study 

o 

o 

o 

o 

o 


engineering 


*9. Please indicate how strongly you disagree or agree with each of the statements: 


Disagree 

Strongly 

Disagree 

Agree 

Agree 

Strongly 

1 prefer 
not to 
answer 


Creative thinking is one of my 
strengths 

o 

o 

o 

o 

o 

1 am skilled at solving problems 
that can have multiDle solutions 

o 

o 

o 

o 

o 
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* 10 . Rate yourself on each of the following traits as compared to your classmates. We 
want the most accurate estimate of how you see yourself. 


Lowest 

Below 


Above 

Highest 

1 prefer 
not to 

10% 

Average 

Average 

Average 

10% 

answer 


Self confidence (social) 

o 

o 

o 

o 

o 

o 

Leadership ability 

o 

o 

o 

o 

o 

o 

Public speaking ability 

o 

o 

o 

o 

o 

o 

Math ability 

o 

o 

o 

o 

o 

o 

Science ability 

o 

o 

o 

o 

o 

o 

Communication skills 

o 

o 

o 

o 

o 

o 

Ability to apply math and science 
principles in solving real world problems 

o 

o 

o 

o 

o 

o 

Business ability 

o 

o 

o 

o 

o 

o 

Ability to perform in teams 

o 

o 

o 

o 

o 

o 

Critical thinking skills 

o 

o 

o 

o 

o 

o 


* 11 . How important do you think each of the following skills and abilities is to becoming a 
successful engineer? 


Not 

Somewhat 

Very 


1 

prefer 
not to 

1 mportant 

1 mtKtrtant 

1 mportant 

Crucial 

answer 


Self confidence (social) 

o 

o 

o 

o 

o 

Leadership ability 

o 

o 

o 

o 

o 

Public speaking ability 

o 

o 

o 

o 

o 

Math ability 

o 

o 

o 

o 

o 

Science ability 

o 

o 

o 

o 

o 

Communication skills 

o 

o 

o 

o 

o 

Ability to apply math and science 
principles in solving real world 
problems 

o 

o 

o 

o 

o 

Business ability 

o 

o 

o 

o 

o 

Ability to perform in teams 

o 

o 

o 

o 

o 

Critical thinking skills 

o 

o 

o 

o 

o 


4C-4 
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* 12 . Please rate your satisfaction with this institution on each aspect of campus life listed 
below. If you do not have experience with this aspect, mark N/ A. 


Very 



Very 


1 

prefer 
not to 

Dissatisfied 

Dissatisfied 

Satisfied 

Satisfied 

N/ A 

answer 


Quality of instruction O O O O O O 


Availability of instructors 

o 

o 

o 

o 

o 

o 

Quality of advising by 
instructors 

o 

o 

o 

o 

o 

o 

Academic advising 

o 

o 

o 

o 

o 

o 


* 13 . Please rate the overall quality of your collegiate experience so far: 

O Very dissatisfied 

o 

Dissatisfied 

o 

Satisfied 

O Very satisfied 
O I prefer not to answer 


* 14 . Think about the engineering, math or science classes you are taking/ have taken 
during the current school year. I ndicate how often you: ( Mark N/ A if you have not taken 
any engineering related classes.) 







prefer 
not to 

Never 

Rarely 

Qccasionally 

Frequently 

N/ A 

answer 


Came late to engineering class 

o 

o 

o 

o 

o 

o 

Skipped engineering class 

o 

o 

o 

o 

o 

o 

Turned in engineering 
assignments that did not reflect 
your best work 

o 

o 

o 

o 

o 

o 

Turned in engineering 
assianments late 

o 

o 

o 

o 

o 

o 
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* 15 . Think about the liberal arts classes (not engineering, math, or science classes)you 
are taking/ have taken during the current school year. I ndicate how often you: (Mark N/ A 
if you have not taken any liberal arts classes.) 







1 

prefer 
not to 

Never 

Rarelir 

Occasionallir 

Frequently 

N/ A 

answer 


Came late to liberal arts class O O O O O O 


Skipped liberal arts class 

o 

o 

o 

o 

o 

o 

Turned in liberal arts assignments 
that did not reflect your best 

work 

o 

o 

o 

o 

o 

o 

Turned in liberal arts assignments 

late 

o 

o 

o 

o 

o 

o 


* 16 . How well are you meeting the workload demands of your coursework? 

o I 

am meeting all of the demands easily 
O I am meeting all of the demands, but it is hard work 
O I am meeting most of the demands, but cannot meet some 

o I 

can meet some of the demands, but cannot meet most 
O I cannot meet any of the demands 
O I prefer not to answer 


* 17 . How stressed do you feel in your coursework right now? 

o 

No stress 

O Moderately low stress 
O Moderate stress 

o 

Moderately high stress 
O High stress 

o 

I prefer not to answer 


4C-6 
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* 18 . During the current school year, how much pressure have you felt with each of the 
following? 


No 

Moderately 

Low 

Moderate 

Moderately 

High 

High 

1 

prefer 
not to 

Pressure 

Pressure 

Pressure 

Pressure 

Pressure 

answer 


Course load (amount of 
course material being 
covered) 

o 

o 

o 

o 

o 

o 

Course pace (the rate at 
which the course material is 
being covered) 

o 

o 

o 

o 

o 

o 

Balance between social and 
academic life 

o 

o 

o 

o 

o 

o 


* 19 . During the current school year, how often have you interacted with your instructors 
(faculty, teaching assistants) in your engineering, math, or science classes (e.g. by 
phone, e-mail, IM, or in person)? Mark N/ A if you have not taken any engineering, math, 
or science classes this year. 



Never 

Rarelv 

Occasionally 

Often 

Very 

Often 

N/ A 

1 

prefer 
not to 
answer 

Instructors during class 

o 

o 

o 

o 

o 

o 

o 

Instructors during office hours 

o 

o 

o 

o 

o 

o 

o 

1 nstructors outside of class or 
office hours 

o 

o 

o 

o 

o 

o 

o 


* 20 . Some people are involved in non-engineering activities on or off campus, such as 
hobbies, civic or church organizations, campus publications, student government, social 
fraternity or sorority, sports, etc. How important is it for you to be involved in these kind 
of activities? 

o 

Not I mportant 

O Somewhat I mportant 

o 

Very I mportant 

o 

Essential 

O I prefer not to answer 
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* 21 . How often are you involved in the kinds of non-engineering activities described 
above? 

o 

Never 
O Rarely 
O Occasionally 
O Frequently 

o 

I prefer not to answer 


* 22 . What is your level of involvement in student engineering activities such as 
engineering societies? 

o 

No involvement 

o 

Limited involvement 

o 

Moderate involvement 
O Extensive involvement 

o 

I prefer not to answer 


* 23 . Since coming to college, have you had any research experience(s)? 

O No 

O Yes, in engineering related areas 
O Yes, in non-engineering related areas 
O Yes, in both engineering and non-engineering related areas 
O I prefer not to answer 


* 24 . Before college, how much knowledge did you have about the engineering profession? 

o 

No knowledge 

o 

Limited knowledge 
O Moderate knowledge 

o 

Extensive knowledge 

o 

I prefer not to answer 


4C-8 
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* 25 . Since entering college, how much knowledge have you gained about the engineering 
profession? 


o 

o 

o 

o 

o 


No knowledge 
Limited knowledge 
Moderate knowledge 
Extensive knowledge 
I prefer not to answer 


* 26 . How did you gain your knowledge about the engineering profession? (Mark all that 
apply) 

□ From being a visitor 

□ From being a co-op student 

□ From being an intern 

□ From being an employee 

□ From a family member 

□ From a close friend 

O Other: I 

□ I prefer not to answer 

* 27 . Do any of your immediate family members (parents, siblings) hold an engineering 
degree? 

O No 

O Yes 

O I prefer not to answer 

* 28 . Do you have any concerns about your ability to finance your college education? 

o 

None (I am confident that I will have sufficient funds) 

O Some (but I probably will have sufficient funds) 

o 

Major (I have funds but will graduate with significant debt) 

O Extreme (not sure if I will have sufficient funds to complete college) 

o 

I prefer not to answer 
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*29. About how many hours do you spend in a typical 7-day week doing each of the 
following? 









More 

than 

1 

prefer 
not to 

0 

1-5 

6-10 

11-15 

16-20 

21-25 

26-30 

30 

answer 


Preparing for class 

(studying, reading, writing, 
doing homework or lab work, 
analyzing data, rehearsing, 
and other academic activities) 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Working for pay 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Participating in co- 
curricular activities 

(organizations, campus 
publications, student 
government, social fraternity 
or sorority, intercollegiate or 
intramural sports, etc.) 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Relaxing and 
socializing (watching TV, 
partying, exercising, etc.) 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Providing care for 
dependents living with you 

(parents, children, spouse, 
etc.) 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Commuting to class 

(driving, walking, etc.) 

o 

o 

o 

o 

o 

o 

o 

o 

o 


*30. What is your expected grade point average this academic term? 

o A or A+ (3.9 or above) 

O (3. 5-3. 8) 

O B-)- (3. 2-3. 4) 

O B (2.9-3. 1) 

O (2. 5-2. 8) 

O c-l- (2. 2-2. 4 ) 

O C (1.9-2. 1) 

O 

or lower (less than 1.5) 

o 

I prefer not to answer 


4C-10 
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*31. What is your expected grade point average overall? 

O A or A+ (3.9 or above) 

O (3. 5-3. 8) 

O B-l- (3. 2-3. 4) 

O B (2.9-3. 1) 

O (2. 5-2. 8) 

O c-l- ( 2 . 2 - 2 . 4) 

O C (1.9-2. 1) 

O 

or lower (less than 1.5) 

o 

I prefer not to answer 

*32. What do you see yourself doing in the first 3 years after graduation? (Mark all that 

Find a job working in the engineering profession 
Find a job working in a non-engineering profession 
Go to graduate school in engineering (Masters/PhD) 

Goto professional school (business, law, medicine, etc.) 

Go to graduate school NOT in engineering (Masters/PhD) 

I 

Other: (please explain) I 
I prefer not to answer 

*33. How sure are you about your plans after graduation? 

o 

Not at all sure 

o 

Somewhat sure 
O Pretty sure 

o 

Absolutely sure 

o 

I prefer not to answer 


apply) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 
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* 34 . Do you see yourself pursuing a career in engineering? 

O Definitely Not 
O Probably Not 
O Not Sure 
O Probably Yes 
O Definitely Yes 

o 

I prefer not to answer 

* 35 . Your sex: 

O Male 

o 

Female 

o 

I prefer not to answer 

* 36 . What is your racial or ethnic identification? (Select only one) 

o 

American Indian or other native person 
O Asian, Asian American, or Pacific Islander 

o 

Black or African American 

o 

White (non-Hispanic) 

O Hispanic or Latino/a 

o 

Multiracial 

O Other: I 

O I prefer not to answer 

* 37 . Are you: 

O A U.S. Citizen 

o 

A Permanent Resident of the U.S. 

O other 

O 

I prefer not to answer 


4C-12 
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* 38 . Were you born in the United States? 


o 

o 

o 


Yes 

If no, at what age did you immigrate to the U.S? 
I prefer not to answer 


* 39 . Did one or more of your parents/ guardians immigrate to the United States? 

o Yes 
O No 

O 

I prefer not to answer 

* 40 . Is English your first language? 

o Yes 
O No 

O 

I prefer not to answer 


* 41 . Are you a first-generation college student (first in your immediate family to attend 
college)? 

o Yes 

O No 

O 

I prefer not to answer 
* 42 . Are you enrolled primarily as a: 

o 

Full-time student 

o 

Part-time student 

o 

I prefer not to answer 


* 43 . Would you describe your family as: (Mark one) 

o 

Low income 

o 

Middle income 
O Upper-middle income 

o 

High income 

o 

I prefer not to answer 
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*44. What is the highest level of education that your mother completed? (Mark one) 

o 

Did not finish high school 

o 

Graduated from high school 

o 

Attended college but did not complete degree 

o 

Completed an Associate degree (AA, AS, etc.) 

o 

Completed a Bachelor degree (BA, BS, etc.) 

o 

Completed a Masters degree (MA, MS, etc.) 

o 

Completed a Doctoral or Professional degree (JD, MD, PhD, etc.) 

o 

Don't know or Not applicable 

o 

I prefer not to answer 

*45. What is the highest level of education that your father completed? (Mark one) 

o 

Did not finish high school 

o 

Graduated from high school 

o 

Attended college but did not complete degree 

o 

Completed an Associate degree (AA, AS, etc.) 

o 

Completed a Bachelor degree (BA, BS, etc.) 

O Completed a Masters degree (MA, MS, etc.) 

o 

Completed a Doctoral or Professional degree (JD, MD, PhD, etc.) 

o 

Don't know or Not applicable 
O I prefer not to answer 
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*46. You have been asked to design a playground. You have a limited amount of time and 
resources to gather information for your design. From the following list, please put a 
check mark next to the FI VE kinds of information you would MOST LI KELY NEED as you 
work on your design: 

□ Availability of materials 

□ Body proportions 

□ Budget 

□ Handicapped accessibility 

□ Information about the area 

□ Labor availability and cost 

□ Legal liability 

□ Maintenance concerns 

□ Material costs 

□ Material specifications 

□ Neighborhood demographics 

□ Neighborhood opinions 
n Safety 

□ Supervision concerns 

□ Technical references 
O Utilities 

□ I prefer not to answer 
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*47. Which of the following statements best describes your situation with respect to an 
engineering major? (Mark one) 

I started at this institution... 

o 

Intending to major in engineering and never doubted the decision. 

O Intending to major in engineering and have/had doubts. 

O Considering engineering, but was open to other majors, too. 

o 

Intending another major, but am now considering engineering. 

O Completely undecided about what my major would be. 

o 

I prefer not to answer 

*48. My decision to CONTI NUE with an engineering major primarily came from: (Mark 
one) 

o 

Not applicable - I am not continuing in an engineering major. 

O Experiences with PRE-ENGI NEERI NG-related (math, physics, etc) coursework, 

O faculty, and/or research/internship(s) 

o Experiences with ENGI NEERI NG-related coursework, faculty, and/or 
O research/internship(s) 

O Experiences with OTHER coursework, faculty, and/or research/internship(s) 

o 

I prefer not to answer 

*49. My DOUBTS about continuing in an engineering major primarily came from: (Mark 
one) 

O Not applicable - I never doubted continuing in an engineering major. 

O Experiences with PRE-ENGI NEERI NG-related (math, physics, etc) coursework, 

O faculty, and/or research/internship(s) 

o Experiences with ENGI NEERI NG-related coursework, faculty, and/or 
O research/internship(s) 

o 

Experiences with OTHER coursework, faculty, and/or research/internship(s) 

o 

I prefer not to answer 
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50. I s there anything you want to tell us about your experiences in engineering that we 
haven't already asked you about? 


*51. How did you learn about this survey? (Mark all that apply) 

□ Advertisement in student newspaper 

□ Announcement in class 

□ Given time in class to complete the survey 

□ Announcement or email from engineering society or other student group 

□ Email from a friend 

□ Email from school official/dean 

□ Poster/flyer 
O Other 

□ I prefer not to answer 

*52. Would you have been willing to take this survey if the compensation was: (Mark all 
that apply) 

□ Drawing for $50 (1 chance out of 100) 

□ $0 - No compensation 

□ I prefer not to answer 
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Help 

General Questions Order Preview Finish 

This is a preview of how this survey wili iook. In the preview the survey navigation buttons are inactive, use the section 
number buttons to view different sections. Some navigation buttons may not appear on your final survey, depending on 
what access it is assigned. The survey wili use the background color of the document in which it is embedded. If you have 

no further changes ciick Finish at the bottom of this page. 

Section: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 


Academic Pathways Of People Learning Engineering Survey 

(APPLES) 

Page 1 of 7 

For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 

*1. What school are you currently attending? 

o INSERT NAME OF SCHOOL 
O Other: | 

O I prefer not to answer 

*2. What is your current academic standing? 
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O Freshman 
O Sophomore 
O Junior 
O Senior 

O Fifth year senior or more 
O Graduate student 
O Other: | 

O I prefer not to answer 


When you entered this institution were you: 

O A first-time coiiege student 
O Returning or non-traditionai coliege student 
O A transfer student from a two-year institution 
O A transfer student from a four-year institution 

O A transfer student from an institution that participates in a 3 -(- 2 engineering program 
O I prefer not to answer 


What were you most interested in majoring in when you first came to university? (Choose one) 

O Arts and Flumanities 
O Engineering 
O Math and Naturai Sciences 
O Physicai Sciences 
O Sociai Sciences 
O Other 

O I prefer not to answer 


What is your current major or first choice of major? (Mark one) 

O Aerospace Engineering 
O Chemicai Engineering 
O Civii Engineering 
O Eiectricai Engineering 
O Industriai Engineering 
O Materiais and Metailurgical Engineering 
O Mechanical Engineering 

O Computer Science/Engineering (in engineering) 
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O Computer Science (non-engineering) 
O Other Engineering: | 

O Arts and Humanities 
O Math and Natural Sciences 
O Physical Sciences 
O Social Sciences 
O Other Non-EngIneerIng: | 

O I prefer not to answer 


What is your second choice of major or second major/minor? 
(Mark one or N/A if not applicable) 

O Aerospace Engineering 
O Chemical Engineering 
O Civil Engineering 
O Electrical Engineering 
O Industrial Engineering 
O Materials and Metallurgical Engineering 
O Mechanical Engineering 

O Computer Science/Engineering (in engineering) 

O Computer Science (non-engineering) 

O Other Engineering: | 

O Arts and Humanities 
O Math and Natural Sciences 
O Physical Sciences 
O Social Sciences 
O Other Non-EngIneerIng: | 

O n/a 
O Undecided 
O I prefer not to answer 


Do you intend to complete a major in engineering? 
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O Definitely not 
O Probably not 
O Not sure 
O Probably yes 
O Definitely yes 
O I prefer not to answer 


* 8 . Do you intend to practice, conduct research in, or teach engineering for at least 3 years after 
graduation? 

O Definitely not 
O Probably not 
O Not sure 
O Probably yes 
O Definitely yes 
O I prefer not to answer 

Page 1 of 7 

Save I Next Page | 


The survey title and other general fields are on the General tab. Individual survey questions are added and modified on the 
Questions tab. Questions may be re-ordered or deleted from the Order tab. You may see a preview of your survey at any 
time, by going to the Preview tab. If you have no further changes click Finish to go back to the Management Interface. 

Click here to open the Help window. 

General | Questions [ Order [ Preview | Finish [ 
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Survey Design 

Help 

General Questions Order Preview Finish 

This is a preview of how this survey wili iook. In the preview the survey navigation buttons are inactive, use the section 
number buttons to view different sections. Some navigation buttons may not appear on your final survey, depending on 
what access it is assigned. The survey wili use the background color of the document in which it is embedded. If you have 

no further changes ciick Finish at the bottom of this page. 

Section: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 


Academic Pathways Of People Learning Engineering Survey 


(APPLES) 

Page 2 of 7 

For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 


* 9 . We are interested in knowing why you are or were studying engineering. Please indicate below 
the extent to which the following reasons apply to you: 



Not a 

Minimai 

Moderate 

I prefer 
Major not to 


Reason 

Reason 

Reason 

Reason answer 

Technoiogy piays an important roie in soiving 

society's probiems 

o 

o 

o 

o 

o 

Engineers make more money than most 
other professionais 

o 

o 

o 

o 

o 

My parent(s) would disapprove if I chose a 
major other than engineering 

o 

o 

o 

o 

o 

Engineers have contributed greatiy to fixing 
probiems in the worid 

o 

o 

o 

o 

o 

Engineers are weii paid 

o 

o 

o 

o 

o 

My parent(s) want me to be an engineer 

o 

o 

o 

o 

o 

An engineering degree wiil guarantee me a 
job when I graduate 

o 

o 

o 

o 

o 

A faculty member, academic advisor, 
teaching assistant or other university 
affiiiated person has encouraged and/or 

o 

o 

o 

o 

o 
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inspired me to study engineering 

A non-university affiliated mentor has 
encouraged and/or inspired me to study 


engineering 

o 

o 

o 

o 

o 

A mentor has introduced me to people and 
opportunities in engineering 

o 

o 

o 

o 

o 

I feei good when I am doing engineering 

o 

o 

o 

o 

o 

I iike to buiid stuff 

o 

o 

o 

o 

o 

I think engineering is fun 

o 

o 

o 

o 

o 

Engineering skiils can be used for the good of 

society 

o 

o 

o 

o 

o 

I think engineering is interesting 

o 

o 

o 

o 

o 

I iike to figure out how things work 

o 

o 

o 

o 

o 


Please indicate how strongly you disagree or agree with each of the statements: 

I prefer 

Disagree Agree not to 

Strongiy Disagree Agree Strongiy answer 


Creative thinking is one of my strengths 

o 

o 

o 

o 

o 

I am skiiled at soiving probiems that can 
have muitipie soiutions 

o 

o 

o 

o 

o 

A mentor has supported my decision to 
major in engineering 

o 

o 

o 

o 

o 


Rate yourself on each of the following traits as compared to your classmates. We want the 
most accurate estimate of how you see yourself. 


I prefer 

Lowest Beiow Above Highest not to 



10% 

Average Average Average 

10% 

answer 

Seif confidence (social) 

O 

o 

o 

o 

o 

o 

Leadership abiiity 

O 

o 

o 

o 

o 

o 

Pubiic speaking abiiity 

O 

o 

o 

o 

o 

o 

Math ability 

O 

o 

o 

o 

o 

o 

Science abiiity 

O 

o 

o 

o 

o 

o 

Communication skilis 

o 

o 

o 

o 

o 

o 

Abiiity to apply math and science 
principles in soiving real world 
problems 

o 

o 

o 

o 

o 

o 

Business ability 

o 

o 

o 

o 

o 

o 

Ability to perform in teams 

o 

o 

o 

o 

o 

o 

Critical thinking skills 

o 

o 

o 

o 

o 

o 
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How important do you think each of the following skills and abilities is to becoming a 
successful engineer? 



Not Somewhat Very 

Important Important Important 

Crucial 

I prefer 
not to 
answer 

Self confidence (social) 

o 

o 

o 

o 

o 

Leadership ability 

o 

o 

o 

o 

o 

Public speaking ability 

o 

o 

o 

o 

o 

Math ability 

o 

o 

o 

o 

o 

Science ability 

o 

o 

o 

o 

o 

Communication skills 

o 

o 

o 

o 

o 

Ability to apply math and science 
principles in solving real world 
problems 

o 

o 

o 

o 

o 

Business ability 

o 

o 

o 

o 

o 

Ability to perform in teams 

o 

o 

o 

o 

o 


Please rate your satisfaction with this institution on each aspect of campus life listed below. 
(Mark N/A if you do not have experience with this aspect.) 



Very 

Dissatisfied 

Dissatisfied Satisfied 

Very 

Satisfied 

N/A 

I prefer 
not to 
answer 

Quality of instruction 

o 

o 

o 

o 

o 

o 

Availability of instructors 

o 

o 

o 

o 

o 

o 

Quality of advising by 
instructors 

o 

o 

o 

o 

o 

o 

Academic advising 

o 

o 

o 

o 

o 

o 


During the current school year, what portion of your classes have used the following teaching 
methods? 





Less 


More 

All or 

I prefer 



Very 

than 

About 

than 

nearly 

not to 


None 

little 

half 

half 

half 

all 

answer 

Individual projects 

o 

o 

o 

o 

o 

o 

o 

Team projects 

o 

o 

o 

o 

o 

o 

o 


Please rate the overall quality of your collegiate experience so far: 
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O Very dissatisfied 
O Dissatisfied 
O Satisfied 
O Very satisfied 
O I prefer not to answer 

Page 2 of 7 

Previous Page | Save | Next Page ] 


The survey title and other generai fields are on the General tab. Individual survey questions are added and modified on the 
Questions tab. Questions may be re-ordered or deleted from the Order tab. You may see a preview of your survey at any 
time, by going to the Preview tab. If you have no further changes click Finish to go back to the Management Interface. 

Click here to open the Help window. 

General | Questions [ Order [ Preview | Finish [ 
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Survey Design 

Help 

General Questions Order Preview Finish 

This is a preview of how this survey wili iook. In the preview the survey navigation buttons are inactive, use the section 
number buttons to view different sections. Some navigation buttons may not appear on your final survey, depending on 
what access it is assigned. The survey wili use the background color of the document in which it is embedded. If you have 

no further changes ciick Finish at the bottom of this page. 

Section: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 


Academic Pathways Of People Learning Engineering Survey 


(APPLES) 

Page 3 of 7 

For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 


* 16 . Think about the engineering, math or science classes you are taking/have taken during the 
current school year. Indicate how often you: 

(Mark N/A if you have not taken any engineering related classes.) 


I prefer 
not to 



Never 

Rareiy 

Occasionaiiy Frequentiy 

N/A 

answer 

Came late to engineering 
ciass 

o 

o 

o 

o 

o 

o 

Skipped engineering ciass 

o 

o 

o 

o 

o 

o 

Turned in engineering 
assignments that did not 
refiect your best work 

o 

o 

o 

o 

o 

o 

Turned in engineering 
assignments iate 

o 

o 

o 

o 

o 

o 
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Think about the liberal arts classes (not engineering, math, or science classes)you are 
taking/have taken during the current school year. Indicate how often you: 

(Mark N/A if you have not taken any liberal arts classes.) 


Came late to liberal arts 

Never 

Rarely 

class 

o 

o 

Skipped liberal arts class 

Turned in liberal arts 
assignments that did not 

o 

o 

reflect your best work 
Turned in liberal arts 

o 

o 

assignments late 

o 

o 


Occasionally Frequently 

N/A 

I prefer 
not to 
answer 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 


How well are you meeting the workload demands of your coursework? 

O I am meeting all of the demands easily 
O I am meeting all of the demands, but it is hard work 
O I am meeting most of the demands, but cannot meet some 
O I can meet some of the demands, but cannot meet most 
O I cannot meet any of the demands 
O I prefer not to answer 


How stressed do you feel in your coursework right now? 

O No stress 
O Moderately low stress 
O Moderate stress 
O Moderately high stress 
O High stress 
O I prefer not to answer 


During the current school year, how much pressure have you felt with each of the following? 



No 

Moderately 

Moderate 

Moderately 

High 

not to 

Course load (amount 
of course material 

Pressure 

Low Pressure 

Pressure 

High Pressure 

Pressure 

answer 

being covered) 

o 

o 

o 

o 

o 

o 


1/30/2008 9:33 AM 


http;//caee-aps. stanford.edu/phpESP/admin/manage.php 


Course pace (the rate 
at which the course 
material is being 


covered) 

o 

o 

o 

o 

o 

o 

Balance between social 
and academic life 

o 

o 

o 

o 

o 

o 


During the current school year, how often have you interacted with your instructors (faculty, 
teaching assistants) in your engineering, math, or science classes (e.g. by phone, e-mail, IM, 
or in person)? 

(Mark N/A if you have not taken any engineering, math, or science classes this year.) 



Never 

Rarely Occasionally 

Often 

Very 

often 

N/A 

I prefer 
not to 
answer 

Instructors during class 

o 

o 

o 

o 

o 

o 

o 

Instructors during office hours 

o 

o 

o 

o 

o 

o 

o 

Instructors outside of class or 

office hours 

o 

o 

o 

o 

o 

o 

o 


Some people are involved in non-engineering activities on or off campus, such as hobbies, civic 
or church organizations, campus publications, student government, social fraternity or 
sorority, sports, etc. How important is it for you to be involved in these kind of activities? 


O Not important 
O Somewhat important 
O Very important 
O Essential 

O / prefer not to answer 


How often are you involved in the kinds of non-engineering activities described above? 

O Never 
O Rarely 
O Occasionally 
O Frequently 
O I prefer not to answer 
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*24. What is your level of involvement in student engineering activities such as engineering clubs or 
societies? 

O No involvement 
O Limited involvement 
O Moderate involvement 
O Extensive involvement 
O I prefer not to answer 


*25. Since coming to college, have you had any research experience(s)? (Mark one) 

O No 

O Yes, in engineering related areas 
O Yes, in non-engineering related areas 
O Yes, in both engineering and non-engineering related areas 
O I prefer not to answer 
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Survey Design 

Help 

General Questions Order Preview Finish 

This is a preview of how this survey wili iook. In the preview the survey navigation buttons are inactive, use the section 
number buttons to view different sections. Some navigation buttons may not appear on your final survey, depending on 
what access it is assigned. The survey wili use the background color of the document in which it is embedded. If you have 

no further changes ciick Finish at the bottom of this page. 
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Academic Pathways Of People Learning Engineering Survey 

(APPLES) 

Page 4 of 7 

For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 

* 26 . Before college, how much knowledge did you have about the engineering profession? 

O No knowiedge 
O Limited knowiedge 
O Moderate knowiedge 
O Extensive knowiedge 
O I prefer not to answer 


* 27 . Since entering college, how much knowledge have you gained about the engineering 
profession? 

O No knowiedge 
O Limited knowiedge 
O Moderate knowiedge 
O Extensive knowiedge 
O I prefer not to answer 
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How much exposure have you had to a professional engineering environment as a visitor, 
intern, or employee? 

O No exposure 
O Limited exposure 
O Moderate exposure 
O Extensive exposure 
O I prefer not to answer 


How did you gain your knowledge about the engineering profession? (Mark all that apply) 

CH From being a visitor 

CH From being a co-op student or intern 

CH From being an employee 

CH From a family member 

CH From a close friend 

CH From schooi-related experiences (i.e., a professor or class) 

CH Other: | 

CH I prefer not to answer 


Do any of your immediate family members (parents, siblings) hold an engineering degree? 

O No 
O Yes 

O I prefer not to answer 


Do you see yourself continuing in an engineering major? 

O No - I am NOT majoring or planning to major in engineering 
O Yes 

O I prefer not to answer 


Do you see yourself pursuing a career in engineering? 
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O Definitely not 
O Probably not 
O Not sure 
O Probably yes 
O Definitely yes 
O I prefer not to answer 


*33. How likely is it that you would do each of the following after graduation? 


I prefer 

Definitely Probably Not Probably Definitely not to 



not 

not 

sure 

yes 

yes 

answer 

Work in an engineering job 

o 

o 

o 

o 

o 

o 

Work in a non-engineering job 

o 

o 

o 

o 

o 

o 

Go to graduate school in an 
engineering discipline 

o 

o 

o 

o 

o 

o 

Go to graduate school outside of 
engineering 

o 

o 

o 

o 

o 

o 


*34. Do you have any concerns about your ability to finance your college education? 

O None (I am confident that I will have sufficient funds) 

O Some (but I probably will have sufficient funds) 

O Major (I have funds but will graduate with significant debt) 

O Extreme (not sure if I will have sufficient funds to complete college) 

O I prefer not to answer 


*35. What is your cumulative grade point average? 


o A or A-l- 

(i.e., 3.9 or above on a 4.0 scale) 

O A- 

(3. 5-3. 8) 

O B+ 

(3. 2-3. 4) 

O B 

(2.9-3. 1) 

O B- 

(2. 5-2. 8) 

O c+ 

(2. 2-2. 4) 

O C 

(1.9-2. 1) 

O C- or lower 

(less than 1.5) 

O I prefer not to answer 
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Survey Design 

Help 
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This is a preview of how this survey wili iook. In the preview the survey navigation buttons are inactive, use the section 
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For best viewing results, please maximize your browser window. 

Questions marked with a * are required. 

*36. Your sex: 

O Female 
O Male 

O I prefer not to answer 

*37. What is your racial or ethnic identification? (Mark all that apply) 

CH American Indian or Alaska Native 

CH Asian or Asian American 

CH Black or African American 

CH Hispanic or Latino/a 

CH Native Hawaiian or Pacific Islander 

CH White 

CH Other: | 
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CH I prefer not to answer 


How old are you? (Mark one) 

O 17 or younger 
O 18-19 
O 20-23 
O 24-29 
O 30-39 
O 40-55 
O over 55 

O I prefer not to answer 


Are you: 

o A U.S. Citizen 

O A Permanent Resident of the U.S. 
O Other 

O I prefer not to answer 


Were you born in the United States? 

O Yes 

O If no, at what age did you immigrate to the U.S: [ 
O I prefer not to answer 


Did one or more of your parents/guardians immigrate to the United States? 

O Yes 
O No 

O I prefer not to answer 


1/30/2008 9:34 AM 


http;//caee-aps. stanford.edu/phpESP/admin/manage.php 


Is English your first language? 

O Yes 
O No 

O I prefer not to answer 


Are you a first-generation college student (first in your immediate family to attend 
college)? 

O Yes 
O No 

O I prefer not to answer 


Are you enrolled primarily as a: 

O Full-time student 
O Part-time student 
O I prefer not to answer 


Which of the following best describes where you are living now while attending college? 

O Dormitory or other campus housing 

O Residence (house, apartment, etc.) within walking distance of the institution 
O Residence (house, apartment, etc.) within driving distance of the institution 
O I prefer not to answer 


Would you describe your family as: (Mark one) 

O High income 
O Upper-middle income 
O Middle Income 
O Lower-middle income 
O Low income 
O I prefer not to answer 
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*47. What is the highest level of education that your mother completed? (Mark one) 

O Did not finish high school 
O Graduated from high school 
O Attended college but did not complete degree 
O Completed an Associate degree (AA, AS, etc.) 

O Completed a Bachelor degree (BA, BS, etc.) 

O Completed a Masters degree (MA, MS, etc.) 

O Completed a Doctoral or Professional degree (JD, MD, PhD, etc.) 

O Don't know or not applicable 
O I prefer not to answer 


*48. What is the highest level of education that your father completed? (Mark one) 

O Did not finish high school 
O Graduated from high school 
O Attended college but did not complete degree 
O Completed an Associate degree (AA, AS, etc.) 

O Completed a Bachelor degree (BA, BS, etc.) 

O Completed a Masters degree (MA, MS, etc.) 

O Completed a Doctoral or Professional degree (JD, MD, PhD, etc.) 

O Don't know or not applicable 
O I prefer not to answer 
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Questions tab. Questions may be re-ordered or deleted from the Order tab. You may see a preview of your survey at any 
time, by going to the Preview tab. If you have no further changes click Finish to go back to the Management Interface. 

Click here to open the Help window. 

General | Questions [ Order [ Preview | Finish [ 
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*49. Of the twenty-three design activities below, please put a check mark next to the SIX MOST 
IMPORTANT. 

CH Abstracting 
CH Brainstorming 
CH Buiiding 
CH Communicating 
CH Decomposing 
CH Evaiuating 
CH Generating aiternatives 
CH Goal setting 
CH Identifying constraints 
CH Imagining 
CH Iterating 
CH Making decisions 
CH Making trade-offs 
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CH Modeling 
CH Planning 
CH Prototyping 
CH Seeking information 
CH Sketching 
CH Synthesizing 
CH Testing 

CH Understanding the problem 
CH Using creativity 
CH Visualizing 
CH I prefer not to answer 
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Academic Pathways Study Fall' 05 Survey 

Large Midwestern Public University (LMPub) 

Please click the SUBMIT button only after you have completed the survey. For best 
viewing results, please maximize your browser window. 


1 . What is your expected year of graduation from college? 

C 2005 
C 2006 
C 2007 
C 2008 
C 2009 
C 2010 
C 2011 

C 2012 or later 

2. Do you intend to complete a major in engineering? 

C Definitely Not 
C Probably Not 
C Not Sure 
C Probably Yes 
C Definitely Yes 

3. What do you intend to major in? 

C Aerospace engineering & mechanics 
C Astrophysics 

C Bio-based products engineering 
C Biomedical engineering 
C Biosystems & agricultural engineering 
C Chemical engineering 
C Chemistry 
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C Civil engineering 
C Computer engineering 
C Computer science 
C Electrical engineering 
C Geological engineering 
C Geology 
C Geophysics 

C Materials science & engineering 
C Mathematics 
C Mechanical engineering 
C Physics 
C Statistics 
C Arts & humanities 
C Education 
C Social Science 

^ Other non-engineering I 


If you intend to DOUBEE MAJOR, what is the second major you intend 
complete? (Mark N/A if you do not intend to double major.) 

C N/A 

C Aerospace engineering & mechanics 
C Astrophysics 

C Bio-based products engineering 
C Biomedical engineering 
C Biosystems & agricultural engineering 
C Chemical engineering 
C Chemistry 
C Civil engineering 
C Computer engineering 
C Computer science 
C Electrical engineering 
C Geological engineering 
C Geology 


C Geophysics 

C Materials science & engineering 
C Mathematics 
C Mechanical engineering 
C Physics 
C Statistics 
C Arts & humanities 
C Education 
C Social Science 

^ Other non-engineering I 


5. Do you intend to practice, conduct research in, or teach engineering for at least 3 
years after graduation? 

C Definitely Not 
C Probably Not 
C Not Sure 
C Probably Yes 
C Definitely Yes 


6. If you are thinking of going to graduate school NOT IN ENGINEERING, please 
mark your most probable area of study. Otherwise, mark N/A. 


c 

Business 

c 

Education 

c 

Medicine 

c 

Eaw 

c 

MA/Ph.D. 

c 

Public Service 

c 

Other 

c 

N/A 


7. We are interested in knowing why you are studying engineering now. Please 
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indicate below the extent to which the following reasons apply to you: 



Not a 
Reason 

Minimal Moderate 
Reason Reason 

Major 

Reason 

Technology plays an important role in 
solving society’s problems 

c 

c 


c 

c 


Engineers make more money than most 
other professionals 

c 

c 


c 

c 


My parent(s) would disapprove if I chose a 
major other than engineering 

c 

c 


c 

c 


Engineers have contributed greatly to fixing 
problems in the world 

c 

c 


c 

c 


Engineers are well paid 

c 

c 


c 

c 


Engineering is an occupation that is 
respected by other people 

c 

c 


c 

c 


My parent(s) want me to be an engineer 

c 

c 


c 

c 


An engineering degree will guarantee me a 

job when I graduate 

c 

c 


c 


c 


Engineers are creative problem solvers 

c 

c 


c 

c 


A faculty member, academic advisor, 
teaching assistant or other university 
affiliated person has encouraged and/or 
inspired me to study engineering 

c 

c 


c 


c 


A non- university affiliated mentor has 
encouraged and/or inspired me to study 

engineering 

c 

c 


c 

c 



8. Please indicate how strongly you disagree or agree with each of the statement: 


Disagree 

Strongly 

Disagree 

Agree 

Agree 

Strongly 

I prefer studying in a group to studying by 

myself 

c 

c 


c 

c 


I prefer working as part of a team to working 

alone 

c 

c 


c 

c 


I get along well with others in study situations 

c 

c 


c 

c 


I am a collaborative person 

c 

c 


c 

c 


Creative thinking is one of my strengths 

c 

c 


c 

c 


I am familiar with what a practicing engineer 

does 

c 

c 


c 

c 


I am skilled at solving problems that can have 

c 

c 


c 

c 
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multiple solutions 


9. Rate yourself on each of the following traits as compared to your classmates. We 
want the most accurate estimate of how you see yourself. (Mark one in each row.) 


Lowest 

10% 

Below 

Average 

Average 

Above 

Average 

Highe: 

10% 

Self confidence (social) C 

C 


c 

C 


c 

Leadership ability C 

c 


c 

c 


c 

Public speaking ability C 

c 


c 

c 


c 

Math ability C 

c 


c 

c 


c 

Science ability C 

c 


c 

c 


c 

Computer skills C 

c 


c 

c 


c 

Communication skills C 

c 


c 

c 


c 

Ability to apply math and science 







principles in solving real world C 

c 


c 

c 


c 

problems 





Business ability C 

c 


c 

c 


c 

Ability to perform in teams C 

c 


c 

c 


c 

Critical Thinking skills C 

c 


c 

c 


c 


10. How important do you think each of the following skills and abilities is to 
becoming a successful engineer? (Mark one in each row.) 


Self confidence (social) 
Leadership ability 
Public speaking ability 
Math ability 
Science ability 
Computer skills 
Communication skills 
Ability to apply math and science 
principles in solving real world 


Not Somewhat Very Cj-udal 
Important Important Important 

c c c c 

c c c c 

c c c c 

c c c c 

c c c c 

c c c c 

c c c c 

c c c c 
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problems 
Business ability 
Ability to perform in teams 


c c c 

c c c 


c 

c 


11 . 


Please rate your satisfaction with this institution on each of the aspects of 
campus life listed below. If you do not have experience with this aspect, mark 


N/A. 



Very 

Dissatisfied 

Dissatisfied 

Satisfied 

Very 

Satisfied 

N/A 

Quality of instruction by 
faculty 

c 


c 


c 


c 


c 

Quality of advising by faculty 
Availability of faculty 

c 

c 


c 

c 


c 

c 


c 

c 


c 

c 

Quality of instruction by 
teaching assistants 

c 


c 


c 


c 


c 

Quality of advising by 
teaching assistants 

c 


c 


c 


c 


c 

Availability of teaching 
assistants 

c 


c 


c 


c 


c 


12. Please rate your satisfaction with each of the following at this institution. If you 
do not use the service or facility, mark N/A. 


Very 

Dissatisfied 

Dissatisfied 

Satisfied 

Very 

Satisfied 

NA 

Computer facilities 

c 

c 


c 

c 


c 

Libraries 

c 

c 


c 

c 


c 

Classrooms 

c 

c 


c 

c 


c 

Tutoring 

c 

c 


c 

c 


c 

Academic advising 

c 

c 


c 

c 


c 

Laboratories 

c 

c 


c 

c 


c 


13. Since the beginning of the Fall term, how often have you taken courses which 
required your engagement in individual and/or group projects? 
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C Never 
C Rarely 
C Oecasionally 
C Frequently 


14. Think about the engineering classes you have taken since the beginning of the 
Fall term (engineering, math, and science classes). Indicate how often you: 
(Mark N/A if you have not taken any engineering related classes.) 



Never 

Rarely 

Occasionally Frequently 

N/A 

Came late to engineering 
class 

c 

c 


c 

c 


c 

Skipped engineering class 

c 

c 


c 

c 


c 

Turned in engineering 
assignments that did not 
reflect your best work 

c 

c 


c 

c 


c 

Turned in engineering 
assignments late 

c 

c 


c 

c 


c 

Thought engineering classes 
were boring 

c 

c 


c 

c 


c 


15. Think about the liberal arts classes you have taken since the beginning of the 
Fall term (not engineering, math, or science classes). Indicate how often you: 
(Mark N/A if you have not taken any non-engineering related classes.) 


Came late to liberal arts class 
Skipped liberal arts class 
Turned in liberal arts 
assignments that did not 
reflect your best work 
Turned in liberal arts 
assignments late 
Thought liberal arts classes 
were boring 


Never 

c 

c 

Rarely 

C 

c 


Occasionally Frequently 

c c 

c c 

N/A 

c 

c 

c 

c 


c 

c 


c 

c 

c 


c 

c 


c 

c 

c 


c 

c 


c 


16. How often have you interacted with the following people since the beginning of 
the Fall term (e.g. by phone, e-mail. Instant Messenger, or in person)? (Mark 
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one for each item.) 


Faculty during class 
Faculty during office hours 
Faculty outside of class or office hours 
Teaching Assistants during class 
Teaching Assistants during office hours 
Teaching Assistants outside of class or 

office hours 


Never 

1-2 1-2 
times times 
per per 
TermMonth 

Once 

per 

Week 

2-3 

times 

per 

Week 

Daily 

c 

c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 



17. What portion of the courses you have taken since the beginning of the Fall term 
have been taught primarily by graduate students? 

C None 
C Very little 
C Less than half 
C About half 
C More than half 
C All or nearly all 


18. Since the beginning of the Fall term, what portion of your classes used the 
following teaching methods? 


None 

Very 

little 

Less 

than 

half 

., More All or 

About , 

, than nearly 

half all 

Lectures C 

C 

c 

c 

c 

c 

Individual Projects C 

c 

c 

c 

c 

c 

Team Projects C 

c 

c 

c 

c 

c 

Labs C 

c 

c 

c 

c 

c 

Seminars C 

c 

c 

c 

c 

c 
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19. To what extent have your courses required your engagement in individual and/or 
group projects? 

C Too Few 
C Enough 
C Too many 

20. Some people are involved in non-engineering activities on or off campus, such 
as hobbies, civic or church organizations, campus publications, student 
government, social fraternity or sorority, sports, etc. How important is it for you 
to be involved in these kind of activities? 

C Not Important 
C Somewhat Important 
C Very Important 
C Essential 

21. How often are you involved in the kinds of non-engineering activities described 
above? 

C Never 
C Rarely 
C Occasionally 
C Erequently 

22. Thinking about your college experience since the beginning of the Eall term, 
please indicate how much pressure you are feeling related to the following: 


No 

Pressure 

Reasonable 

Pressure 

Extreme 

Pressure 

Course load (amount of course material being ^ 

covered) 

C 


C 

Course pace (the pace at which the course ^ 
material is being covered) 

c 


c 

Balance between social and academic life C 

c 


c 
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23. How well are you meeting the workload demands of your coursework? 

C I am meeting all of the demands easily 
C I am meeting all of the demands, but it is hard work 
C I am meeting most of the demands, but cannot meet some 
C I can meet some of the demands, but cannot meet most 
C I cannot meet any of the demands 

24. How stressed do you feel in your coursework right now? 

C No stress 
C Some stress 
C Reasonable stress 
C Significant stress 
C Extreme stress 

25. Do you have any concern about your ability to finance your college education? 

C None (I am confident that I will have sufficient funds) 

C Some (but I probably will have sufficient funds) 

C Major (not sure if I will have sufficient funds to complete college) 

26. How do you meet your college expenses? 


None 

Very 

little 

Eess 

than 

half 

Aboul 

half 

More All or 
than nearly 
half all 

Self (income) C 

c 

c 

c 

c 

c 

Self (savings) C 

c 

c 

c 

c 

c 

Parents and family C 

c 

c 

c 

c 

c 

Employer support C 

c 

c 

c 

c 

c 

Scholarships and grants C 

c 

c 

c 

c 

c 

Eoans C 

c 

c 

c 

c 

c 
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27. Do you have family members who are practicing engineers? 

C Yes 
C No 

28. Do you have close friends who are practicing engineers? 

C Yes 
C No 

29. How much exposure have you had to a professional engineering environment as 
a visitor, intern, or employee? 

C No exposure 
C Limited exposure 
C Moderate exposure 
C Extensive exposure 

30. About how many hours do you spend in a typical 7-day week doing each of the 
following? 



0 

1-5 

6-10 

11- 

15 

16- 

20 

21- 

25 

26- 

30 

more 

than 

30 

Preparing for class (studying, reading, 
writing, doing homework or lab work, 
analyzing data, rehearsing, and other 
academic activities) 

c 

c 

c 

c 

c 

c 

c 

C 

Working for pay C 

c 

c 

c 

c 

c 

c 

c 

Participating in co-curricular activities 
(organizations, campus publications, 
student government, social fraternity or 
sorority, intercollegiate or intramural 

sports, etc.) 

c 

c 

c 

c 

c 

c 

c 

c 

Relaxing and socializing (watching 
TV, partying, exercising, etc.) 

c 

c 

c 

c 

c 

c 

c 

c 

Providing care for dependents living 
with you (parents, children, spouse. 

c 

c 

c 

c 

c 

c 

c 

c 
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etc.) 

Commuting to class (driving, walking, 

etc.) ^ 


3 1 . Please rate the overall quality of your collegiate experience so far: 

C Very dissatisfied 
C Dissatisfied 
C Satisfied 
C Very satisfied 



33. Did your summer experience advance your interest in studying engineering? 

C Yes 
C No 

34. Did you participate over the summer in any of the following? (Mark all that 
apply.) 

I~ Engineering related intemship/job 
l~ Engineering related research 
l~ Engineering related coursework 

r N/A 
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35. 


In the space provided, list 5 terms you would use to describe “engineering”: 



36. In the space provided, list 5 terms you would use to describe “design”: 


37. In the space provided, list 5 activities you think engineers do at work. 


38. Of the 20 items below, please put a check mark next to the five you think are 
MOST IMPORTANT for practicing engineers. 

[~ Business knowledge 
V~ Communication 

Conducting experiments 
l~ Contemporary issues 
n Creativity 
r~ Data analysis 
f~ Design 

r~ Engineering analysis 
f~ Engineering tools 
r~ Ethics 

Global context 
f~ Eeadership 
[~ Eife-long learning 
r~ Management skills 
f~ Math 

[~ Problem solving 
f~ Professionalism 
r~ Science 
I” Societal context 
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r~ Teamwork 


39. Your sex: 

C Female 
C Male 

40. Please indicate your ethnic background: (Mark all that apply) 

C White/Caucasian 
C African American/Black 
C American Indian/ Alaska Native 
C Asian American/Asian 
C Native Hawaiian/Pacific Islander 
C Mexican American/Chicano 
C Puerto Rican 
C Other Latino 
C Other 

4 1 . Citizenship status : 

I~ U.S. Resident 

I~ Permanent resident (Green card) 

I~ Neither 

42. Do any of your immediate family members hold an engineering degree? (Mark 
all that apply) 

r No 

l~ Yes, both parents 
l~ Yes, father only 
l~ Yes, mother only 
l~ Yes, sibling(s) 
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43. What is the highest level of education that your mother completed? (Mark one 
box) 

l~ Did not finish high school 

l~ Graduated from high school 

l~ Attended college but did not ocmplete degree 

l~ Completed an Associate's degree (A. A., A.S., etc.) 

I~ Completed a Bachelor's degree (B.A., B.S., etc.) 

I~ Completed a Master's degree (M.A., M.S., etc.) 

I~ Completed a Professional degree (J.D., M.D., etc.) 

I~ Completed a Doctoral degree (Ph.D., Ed.D) 


44. What is the highest level of education that your father completed? (Mark one 
box) 

[~ Did not finish high school 
[~ Graduated from high school 
l~ Attended college but did not complete degree 
f~ Completed an Associate's degree (A. A., A.S., etc.) 
r~ Completed a Bachelor's degree (B.A., B.S., etc.) 
n Completed a Master's degree (M.A., M.S., etc.) 
n Completed a Professional degree (J.D., M.D., etc.) 
f~ Completed a Doctoral degree (Ph.D., Ed.D) 


45. What is your best estimate of your parents’ total income last year? Consider 
income from all sources before taxes. (Mark one) 

C Eess than $10,000 
C $10,000-14,999 
C $15,000-19,999 
C $20,000-24,999 
C $25,000-29,999 
C $30,000-39,999 
C $40,000-49,999 
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C $50,000-59,999 
C $60,000-74,999 
C $75,000-99,999 
C $100,000-149,999 
C $150,000-199,999 
C $200,000-249,999 
C $250,000 or more 
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Academic Pathways Study Spring'06 Survey 


Large Midwestern Public University (LMPub) 

Please click the SUBMIT button only after you have completed the survey. For best 
viewing results, please maximize your browser window. 


1 . What is your expected year of graduation from college? 

C 2006 

C 2007 
C 2008 
C 2009 
C 2010 
C 2011 
C 2012 
C 2013 or later 

2. Do you intend to complete a major in engineering? 

C Definitely Not 
C Probably Not 
C Not Sure 
C Probably Yes 
C Definitely Yes 

3. What do you intend to major in? 

C Aerospace engineering & mechanics 
C Astrophysics 

C Bio-based products engineering 
C Biomedical engineering 
C Biosystems & agricultural engineering 
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C Chemical engineering 
C Chemistry 
C Civil engineering 
C Computer engineering 
C Computer science 
C Electrical engineering 
C Petroleum engineering 
C Geological engineering 
C Geology 
C Geophysics 

C Materials science & engineering 
C Mathematics 
C Mechanical engineering 
C Physics 
C Statistics 
C Arts & humanities 
C Education 
C Social science 

^ Other non-engineering I 


If you intend to DOUBEE MAJOR, what is the second major you intend 
complete? (Mark N/A if you do not intend to double major.) 

C Aerospace engineering & mechanics 
C Astrophysics 

C Bio-based products engineering 
C Biomedical engineering 
C Biosystems & agricultural engineering 
C Chemical engineering 
C Chemistry 
C Civil engineering 
C Computer engineering 
C Computer science 
C Electrical engineering 


C Petroleum engineering 
C Geological engineering 
C Geology 
C Geophysics 

C Materials science & engineering 
C Mathematics 
C Mechanical engineering 
C Physics 
C Statistics 
C Arts & humanities 
C Education 
C Social science 

^ Other non-engineering I 


5. Do you intend to practice, conduct research in, or teach engineering for at least 3 
years after graduation? 

C Definitely Not 
C Probably Not 
C Not Sure 
C Probably Yes 
C Definitely Yes 


6. If you are thinking of going to graduate school NOT IN ENGINEERING, please 
mark your most probable area of study. Otherwise, mark N/A. 

C Business 
C Education 
C Medicine 
C Eaw 
C MA/Ph.D. 

C Public Service 
C Other 
C N/A 
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7. We are interested in knowing why you are studying engineering now. Please 
indicate below the extent to which the following reasons apply to you: 


Technology plays an important role in solving 

society’s problems 
Engineers make more money than most other 

professionals 

My parent(s) would disapprove if I chose a 
major other than engineering 
Engineers have contributed greatly to fixing 
problems in the world 
Engineers are well paid 

Engineering is an occupation that is respected 

by other people 
My parent(s) want me to be an engineer 
An engineering degree will guarantee me a job 

when I graduate 
Engineers are creative problem solvers 
A faculty member, academic advisor, teaching 
assistant or other university affiliated person 
has encouraged and/or inspired me to study 

engineering 

A non-university affiliated mentor has 
encouraged and/or inspired me to study 

engineering 


Not a 
Reason 

MinimalModerate 
Reason Reason 

Major 

Reason 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


8. Please indicate how strongly you disagree or agree with each of the statements: 


Disagree 

Strongly 

Disagree Agree 

Agree 

Strongly 

I prefer studying in a group to studying by 

myself 

c 

C 

c 

c 

I am a competitive person 

c 

c 

c 

c 

I prefer working as part of a team to working 

alone 

c 

c 

c 

c 

I get along well with others in study situations 

c 

c 

c 

c 

I strive to get higher grades than my classmates 

c 

c 

c 

c 
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The educational institution I am attending 
promotes competitive work 
My instructors often remind students that they 
need to do better than other students to obtain 

high grades 

I have easy access to work spaces where I can 
participate in peer study/discussion sessions 
with my fellow students 
I am encouraged by my instructors to initiate or 
participate in peer study sessions with my 

fellow students 
I am a collaborative person 
The educational institution I am attending 
promotes collaborative work 


c c 

c c 

c c 

c c 

c c 

c c 


c c 

c c 

c c 

c c 

c c 

c c 


9. Please indicate how strongly you disagree or agree with each of the statements: 


Disagree 

Strongly 

Disagree 

Agree 

Agree 

Strongly 

Creative thinking is one of my strengths C 

C 

C 

c 

I am familiar with what a practicing engineer pi 

does 

c 

c 

c 

I am skilled at solving problems that can have ^ 

multiple solutions 

c 

c 

c 


10. Rate yourself on each of the following traits as compared to your classmates. We 
want the most accurate estimate of how you see yourself. (Mark one in each row.) 


Lowest 

Below 


Above 

Highe: 

10% 

Average 

Average 

Average 

10% 

Self confidence (social) C 

C 


c 

C 


c 

Leadership ability C 

c 


c 

c 


c 

Public speaking ability C 

c 


c 

c 


c 

Math ability C 

c 


c 

c 


c 

Science ability C 

c 


c 

c 


c 

Computer skills C 

c 


c 

c 


c 

Communication skills C 

c 


c 

c 


c 

Ability to apply math and science C 

c 


c 

c 


c 
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principles in solving real world 
problems 
Business ability 
Ability to perform in teams 
Critical Thinking s ki lls 


c 

c 

c 


c 

c 

c 


c 

c 

c 


c 

c 

c 


c 

c 

c 


1 1 . How important do you think each of the following skills and abilities is to 
becoming a successful engineer? (Mark one in each row.) 


Not Somewhat Very 
Important Important Important 

Self confidence (social) □ □ □ 

Crucial 

c 

Leadership ability 

c 

c 


c 

c 


Public speaking ability 

c 

c 


c 

c 


Math ability 

c 

c 


c 

c 


Science ability 

c 

c 


c 

c 


Computer skills 

c 

c 


c 

c 


Communication skills 

c 

c 


c 

c 


Ability to apply math and science 
principles in solving real world 
problems 

c 

c 


c 

c 


Business ability 

c 

c 


c 

c 


Ability to perform in teams 

c 

c 


c 

c 



12. Please rate your satisfaction with this institution on each of the aspects of campus 
life listed below. If you do not have experience with this aspect, mark N/A. 


Very 

Dissatisfied 

Dissatisfied 

Satisfied 

Very 

Satisfied 

N/A 

Quality of instruction by faculty 

C 

C 


C 

C 


c 

Quality of advising by faculty 

c 

c 


c 

c 


c 

Availability of faculty 

c 

c 


c 

c 


c 

Quality of instruction by 
teaching assistants 

c 

c 


c 

c 


c 

Quality of advising by teaching 
assistants 

c 

c 


c 

c 


c 
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Availability of teaching 
assistants 


c c c c c 


13. Please rate your satisfaction with each of the following at this institution. If you 
do not use the service or facility, mark N/A. 


Very 

Dissatisfied 

Dissatisfied 

Satisfied 

Satisfied 

Computer facilities C 

c 


c 

c 

c 

Libraries C 

c 


c 

c 

c 

Classrooms C 

c 


c 

c 

c 

Tutoring C 

c 


c 

c 

c 

Academic advising C 

c 


c 

c 

c 

Laboratories C 

c 


c 

c 

c 


14. Since the beginning of the Spring term, how often have you taken courses which 
required your engagement in individual and/or group projects? 

C Never 
C Rarely 
C Occasionally 
C Frequently 


15. Think about the engineering classes you have taken since the beginning of the 
Spring term (engineering, math, and science classes). Indicate how often you: 
(Mark N/A if you have not taken any engineering related classes.) 


Never 

Came late to engineering class C 

Skipped engineering class C 


Rarely 

C 

c 


Occasionally Frequently 

c c 

c c 

N/A 

c 

c 

Turned in engineering 
assignments that did not reflect C 
your best work 


c 


c 

c 


c 

Turned in engineering ^ 
assignments late 


c 


c 

c 


c 

Thought engineering classes C 


c 


c 

c 


c 


7 


were boring 


16. Think about the liberal arts classes you have taken since the beginning of the 

Spring term (not engineering, math, and science). Indicate how often you: (Mark 
N/A if you have not taken any non-engineering related classes.) 


Never 

Came late to liberal arts class C 
Skipped liberal arts class C 


Rarely 

C 

c 


Occasionally Frequently 

c c 

c c 

N/A 

c 

c 

Turned in liberal arts 
assignments that did not reflect C 
your best work 


c 


c 

c 


c 

Turned in liberal arts ^ 
assignments late 


c 


c 

c 


c 

Thought liberal arts classes 
were boring 


c 


c 

c 


c 


17. How often have you interacted with the following people since the beginning of 
the Spring term (e.g. by phone, e-mail. Instant Messenger, or in person)? (Mark 
one for each item.) 



1-2 1-2 
times times 
per per 
TermMonth 

Once 

2-3 

times 

per 

‘Week 


Never 

per 

Week 

Daily 

Faculty during class C 

c 

c 

c 

c 

c 


Faculty during office hours C 

c 

c 

c 

c 

c 


Faculty outside of class or office hours C 

c 

c 

c 

c 

c 


Teaching Assistants during class C 

c 

c 

c 

c 

c 


Teaching Assistants during office hours C 

c 

c 

c 

c 

c 


Teaching Assistants outside of class or ^ 

office hours 

c 

c 

c 

c 

c 



18. What portion of the courses you have taken since the beginning of the Spring term 
have been taught primarily by graduate students? 

C None 
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C Very little 
C Less than half 
C About half 
C More than half 
C All or nearly all 


19. Sinee the beginning of the Spring term, what portion of your elasses have used the 
following teaching methods? 


None 

Very 

little 

Eess 

than 

half 

More All or 
About , 

, than nearly 

half all 

Eectures C 

C 

c 

c 

c 

c 

Individual Projects C 

c 

c 

c 

c 

c 

Team Projects C 

c 

c 

c 

c 

c 

Eabs C 

c 

c 

c 

c 

c 

Seminars C 

c 

c 

c 

c 

c 


20. To what extent have your courses required your engagement in individual and/or 
group projects? 

C Too Few 
C Enough 
C Too many 

21. Some people are involved in non-engineering activities on or off campus, such as 
hobbies, civic or church organizations, campus publications, student government, 
social fraternity or sorority, sports, etc. How important is it for you to be involved 
in these kind of activities? 

C Not Important 
C Somewhat Important 
C Very Important 
C Essential 
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22. How often are you involved in the kinds of non-engineering activities described 
above? 

C Never 
C Rarely 
C Occasionally 
C Frequently 

23. Thinking about your college experience since the beginning of the Spring term, 
please indicate how much pressure you are feeling related to the following: 


No 

Pressure 

Reasonable 

Pressure 

Extreme 

Pressure 

Course load (amount of course material being ^ 

covered) 

c 


c 

Course pace (the pace at which the course material ^ 

is being covered) 


c 


c 

Balance between social and academic life C 

c 


c 


24. How well are you meeting the workload demands of your coursework? 

C I am meeting all of the demands easily 
C I am meeting all of the demands, but it is hard work 
C I am meeting most of the demands, but cannot meet some 
C I can meet some of the demands, but cannot meet most 
C I cannot meet any of the demands 

25. How stressed do you feel in your coursework right now? 

C No stress 
C Some stress 
C Reasonable stress 
C Significant stress 
C Extreme stress 
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26. Do you have any concerns about your ability to finance your college education? 

C None (I am confident that I will have sufficient funds) 

C Some (but I probably will have sufficient funds) 

C Major (not sure if I will have sufficient funds to complete college) 

27. How do you meet your college expenses? 


None^®"y 

Less 

Aboui 
than , 

, half 

half 

More All or 
than nearly 
half all 

Self (income) C C 

c c 

c 

c 

Self (savings) C C 

c c 

c 

c 

Parents and family C C 

c c 

c 

c 

Employer support C C 

c c 

c 

c 

Scholarships and grants C C 

c c 

c 

c 

Loans C C 

c c 

c 

c 


28. Do you have close friends who are practicing engineers? 

C No 
C Yes 

29. Do you have family members who are practicing engineers? 

C No 
C Yes 

30. How much exposure have you had to a professional engineering environment as a 
visitor, intern, or employee? 

C No exposure 
C Limited exposure 
C Moderate exposure 
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C Extensive exposure 


31. About how many hours do you spend in a typieal 7-day week doing eaeh of the 
following? 


0 

1-5 

6-10 

Il- 

16- 

21- 

26- 

more 

than 

30 

ls 

20 

25 

30 

Preparing for class (studying, reading, 
writing, doing homework or lab work, ^ 
analyzing data, rehearsing, and other 
academic activities) 

C 

c 

c 

c 

c 

c 

c 

Working for pay C 

c 

c 

c 

c 

c 

c 

c 

Participating in co-curricular activities 
(organizations, campus publications, 
student government, social fraternity or C 
sorority, intercollegiate or intramural 

c 

c 

c 

c 

c 

c 

c 

sports, etc.) 








Relaxing and socializing (watching TV, ^ 
partying, exercising, etc.) 

c 

c 

c 

c 

c 

c 

c 

Providing care for dependents living ^ 
with you (parents, children, spouse, etc.) 

c 

c 

c 

c 

c 

c 

c 

Commuting to class (driving, walking, 

etc.) ^ 

c 

c 

c 

c 

c 

c 

c 


32. Please rate the overall quality of your collegiate experience so far: 

C Very dissatisfied 
C Dissatisfied 
C Satisfied 
C Very satisfied 

33. Some students during their academic career have a specific experience that 
prompts them to doubt their decision to major in engineering. Have you had any 
such experiences? 

C No 
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C Yes 


34. If YES, please indieate the type(s) of reason(s) and/or experience(s) that prompted 
you to DOUBT your decision to major in engineering. Check all that apply. 

1~ Assignment/test/exam grade in a math or science class 

l~ Assignment/test/exam grade in an engineering-related class 

l~ Workload-related experience 

l~ Course material in a math or science class 

I~ Course material in an engineering-related class 

l~ Course instruction 

l~ Interaction with peers (e.g. group project, in-class activities, etc.) 

I~ Interaction with a faculty member or instructor 

N/A - 1 have not had any experiences that prompted me to doubt my decision 
to maj or in engineer ing. 

^ Other: I 

35. Please indicate the type(s) of reason(s) and/or experience(s) that prompted you to 
CONFIRM your decision to major in engineering. Check all that apply. 

I~ Assignment/test/exam grade in a math or science class 

l~ Assignment/test/exam grade in an engineering-related class 

l~ Workload-related experience 

l~ Course material in a math or science class 

l~ Course material in an engineering-related class 

l~ Course instruction 

l~ Interaction with peers (e.g. group project, in-class activities, etc.) 

I~ Interaction with a faculty member or instructor 
|— N/A - 1 have not had any experiences that prompted me to confirm my 
decisio n to major in engineering. 

^ Other: I 

36. Since coming to college, have you had any research experience(s)? 
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C No 

C Yes, in engineering related areas 
C Yes, in non-engineering related areas 


37 . 


Your sex: 

C Male 
C Female 


38. Please indicate your ethnic background: (Mark all that apply) 

l~ White/Caucasian 
l~ African American/Black 
l~ American Indian/Alaska Native 
l~ Asian American/Asian 
l~ Native Hawaiian/Pacific Islander 
[~ Mexican American/Chicano 
l~ Puerto Rican 
l~ Other Latino 
l~ Other 

39. Citizenship Status: 

C U.S. Resident 

C Permanent Resident (Green Card) 

C Neither 


40. What was your average grade in high school? (Mark one) 

C A or A-i- 
C A- 
C B-i- 

C B 
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C B- 
C C+ 

C C 

C C- 

C D 


41. What is the highest level education that your mother completed? (Mark one) 

C Did not finish high school 
C Graduated from high school 
C Attended college but did not complete degree 
C Completed an Associate's degree (A.A., A.S., etc.) 

C Completed a Bachelor's degree (B.A., B.S., etc.) 

C Completed a Master's degree (M.A., M.S., etc.) 

C Completed a Professional degree (J.D., M.D., etc.) 

C Complete a Doctoral degree (Ph.D.) 


42. What is the highest level education that your father completed? (Mark one) 

C Did not finish high school 
C Graduated from high school 
C Attended college but did not complete degree 
C Completed an Associate's degree (A. A., A.S., etc.) 

C Completed a Bachelor's degree (B.A., B.S., etc.) 

C Completed a Master's degree (M.A., M.S., etc.) 

C Completed a Professional degree (J.D., M.D., etc.) 

C Complete a Doctoral degree (Ph.D.) 


43. What is your best estimate of your parents’ total income last year? Consider 
income from all sources before taxes. (Mark one) 

C Less than $10,000 
C $10,000-14,999 
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C $15,000-19,999 
C $20,000-24,999 
C $25,000-29,999 
C $30,000-39,999 
C $40,000-49,999 
C $50,000-59,999 
C $60,000-74,999 
C $75,000-99,999 
C $100,000-149,999 
C $150,000-199,999 
C $200,000-249,999 
C $250,000 or more 


44. How many years of college did you complete before you transferred to the 
LMPub? 

C Not applicable - 1 did not transfer to LMPub. 

C None 

C One year completed 
C Two years completed 
C Three years completed 
C Four years completed 
C More than four years completed 


45. What type of institution did you attend before you transferred to LMPub? 

C Not applicable - 1 did not transfer to LMPub. 

C 2-year college 

C 4-year public college/university (other than LMPub) 

C 4-year private college/university (e.g., other school names) 

C Another LMPub campus 

C Another college on the “Big City” campus of LMPub 

^ Other: I 
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46. Do any of your immediate family members hold an engineering degree? (Mark all 
that apply) 

r No 

l~ Yes, both parents 
l~ Yes, father only 
l~ Yes, mother only 
l~ Yes, siblings 


47. Please rate the extent to which you agree that each of the following is a reason 
why you are currently majoring in or considering majoring in engineering: 


Strongly Moderately 
Disagree Disagree 

Disagree 

Unsure 

Agree 

Moderately 

Agree 

Strongly 

Agree 

I think 

engineering ^ 
is 

interesting 

c 

c 

c 

c 

C 

c 

I am 
majoring in 
engineering C 
for my own 
good 

c 

c 

c 

c 

c 

c 

I am 

supposed to ^ 
major in 
engineering 

c 

c 

c 

c 

c 

c 

There may 
be good 
reasons to 
major in 

engineering, C 
but 

personally, I 
don’t see 
any 

c 

c 

c 

c 

c 

c 

I think 

engineering C 
is pleasant 

c 

c 

c 

c 

c 

c 

I think 

engineering C 
is good for 

c 

c 

c 

c 

c 

c 
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me 


Majoring in 
engineering 
is 

something 
that I have 
to do 
I am 
majoring in 
(considering 
majoring in) 
engineering, 
but I am not 
sure if it is 
worth it 
Majoring in 
engineering 
is fun 
It is my 
personal 
decision 
I don’t have 
any choice 
I don’t 
know. I 
don’t see 
what the 
activity 
brings me 
I feel good 
when I am 
doing 
engineering 
activities 
I believe 
engineering 
is important 
for me 
I feel that I 
have to do it 
I am doing 
it, but am 
not sure it is 
a good thing 
to pursue 




c 



c 


c c c c c c c 

c c c c c c c 

c c c c c c c 

c c c c c c c 


c c c c c c c 


c c c c c c c 

c c c c c c c 

c c c c c c c 
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48. Of the twenty-three design activities below, please put a check mark next to the 
SIX MOST IMPORTANT. 

I~ Abstracting 
l~ Brainstorming 
l~ Building 
l~ Communicating 
l~ Decomposing 
l~ Evaluating 
l~ Generating alternatives 
l~ Goal Setting 
l~ Identifying constraints 
l~ Imagining 
l~ Iterating 
l~ Making decisions 
l~ Making trade-offs 
l~ Modeling 
l~ Planning 
l~ Prototyping 
l~ Seeking information 
l~ Sketching 
l~ Synthesizing 
l~ Testing 

l~ Understanding the problem 
l~ Using creativity 
l~ Visualizing 


49. Of the twenty-three design activities below, please put a check mark next to the 
SIX LEAST IMPORTANT. 

f~ Abstracting 
r~ Brainstorming 
f~ Building 
r~ Communicating 


19 


[~ Decomposing 
f~ Evaluating 
[~ Generating alternatives 
n Goal Setting 
n Identifying constraints 
[~ Imagining 
f~ Iterating 
r~ Making decisions 
I~ Making trade-offs 
l~ Modeling 
[~ Planning 
Prototyping 

1~ Seeking information 
n Sketching 
l~ Synthesizing 
r~ Testing 

f~ Understanding the problem 
r~ Using creativity 
[~ Visualizing 

50. For the following engineering design activities, please indicate your level of 
confidence. For example, if you have little or no confidence in your ability to 
model engineering solutions, then mark poor. If you are extremely confident in 
your ability, mark excellent. 


Poor 

Fair 

Good 

Very 

Good 

1 Excellent 

Defining what the problem really is C 

c 

c 

c 

c 

Searching for and collecting information needed to 

n 

n 

c 

n 

solve the problem 



b_^ 

Thinking up potential solutions to the problem C 

c 

c 

c 

c 

Detailing how to build the solution to the problem C 

c 

c 

c 

c 

Assessing and passing judgment on a possible or ^ 

n 

n 

n 

n 

planned solution to the problem 

L_J 

L_J 

L_J 

L_J 

Comparing and contrasting two solutions to the ^ 




n 

problem on a particular dimension such as cost 

L_J 

L_J 

b^ 

L_J 
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Selecting one idea or solution to the problem from 

among those considered 
Communicating elements of the design in 
sketches, diagrams, lists, and written or oral 

reports 


c c c c c 

c c c c c 


5 1 . For the following engineering design activities, please indicate how often you 
engaged in the activity in your coursework in the current academic year. 


Never 

1-2 

times 

per 

term 

1-2 

times 

a 

month 

Once 

a 

week 

2-3 

times 

a 

week 

Daily 

Defining what the problem really is C 

c 

c 

c 

c 

c 

Searching for and collecting information ^ 
needed to solve the problem 

c 

c 

c 

c 

c 

Thinking up potential solutions to the problem C 

c 

c 

c 

c 

c 

Detailing how to build the solution to the ^ 

problem 

c 

c 

c 

c 

c 

Assessing and passing judgment on a possible 
or planned solution to the problem 

c 

c 

c 

c 

c 

Comparing and contrasting two solutions to 
the problem on a particular dimension such as C 

cost 

c 

c 

c 

c 

c 

Selecting one idea or solution to the problem ^ 
from among those considered 

c 

c 

c 

c 

c 

Communicating elements of the design in 
sketches, diagrams, lists, and written or oral C 

reports 

c 

c 

c 

c 

c 


52. For the following engineering design activities, please indicate how well you 

think your courses are preparing you to engage in the activity. For example, if you 
think they are not preparing you at all, then mark poor. If you think they are 
preparing you extremely well, then mark excellent. 



Poor 

Fair 

Well 

Very 

well 

Excellent 

Defining what the problem really is 

C 

c 

C 

C 

C 

Searching for and collecting information needed to 

solve the problem 

c 

c 

c 

c 

c 


21 










Thinking up potential solutions to the problem G G G G 

Detailing how to build the solution to the problem G G G G 

Assessing and passing judgment on a possible or ^ ^ ^ 

planned solution to the problem 

Comparing and contrasting two solutions to the ^ ^ ^ ^ 

problem on a particular dimension such as cost 
Selecting one idea or solution to the problem from 

among those considered 
Communicating elements of the design in sketches, 
diagrams, lists, and written or oral reports 


G 

G 

G 

G 

G 

G 


53. From the following list, please put a check mark next to the FIVE kinds of 
information you would MOST LIKELY NEED as you work on a typical 
engineering problem. 

[~ Problem scope and severity 
r~ Specifications and requirements 
f~ Legal, regulatory, and industry standards 
l~ Risks and safety 
f~ Available budget 
I” Project costs 
[~ Materials 
V~ Labor 
f~ Maintenance 
r~ Schedule and deadlines 
f~ Project and team coordination 
l~ User demographics and opinion 
n User behavior 
f Client who hired engineers 
f~ Other stakeholders (non-user, non-client) 
r~ Impact on natural environment 
Social and physical context 
f Anticipated benefits 
[~ Aesthetics 

f State of the art in engineering and technology 
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54. From the following list, please put a check mark next to the FIVE kinds of 
information you would LEAST LIKELY NEED as you work on a typical 
engineering problem. 

f~ Problem scope and severity 
r~ Specifications and requirements 

Legal, regulatory, and industry standards 
I~ Risks and safety 
f~ Available budget 
n Project costs 
f~ Materials 
[~ Labor 
f~ Maintenance 
n Schedule and deadlines 
[~ Project and team coordination 
r~ User demographics and opinion 
[~ User behavior 
r~ Client who hired engineers 
[~ Other stakeholders (non-user, non-client) 

Impact on natural environment 
Social and physical context 
n Anticipated benefits 
f~ Aesthetics 

State of the art in engineering and technology 


55. Please rank the following items in terms of how important you think they are to 
engineering problem-solving. Eor each of your answers, mark a rank between 1 
and 6. A rank of "1" indicates the most important item and "6" indicates the least 
important item. Use each number only once. 


1 

2 

3 4 

5 6 

Economic context C 

c 

c c 

c c 

Global context C 

c 

c c 

c c 

Natural context C 

c 

c c 

c c 

Societal context C 

c 

c c 

c c 

Technological context C 

c 

c c 

c c 
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Political context G G G G 


G G 


56. What is your expected grade point average this academic term? 

G A or A+ (3. 9-4.0) 

G A-(3.5-3.8) 

G B-t(3.2-3.4) 

G B (2.9-3. 1) 

G B-(2.5-2.8) 

G C-i- (2.2-2.4) 

G C(1.9-2.1) 

G C-(1.5-1.8) 

G D (less than 1 .4) 
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Academic Pathways Study of Engineering Education; Large Midwestern Public University 

Focus Group Discussion Guide (April 5, 2007) 

Introduction 

Good evening, thanks for taking time to help us with this study. 

My name is [name of researcher] and I am working for the [name of department], along with 
engineering schools at four other universities around the country. We are studying the 
experiences of engineering students so we can find ways to improve the education process. 

You were invited because, obviously, you are engineering students and we want to hear about 
your experiences and your opinions of your education programs. 

There are no right or wrong answers and we expect you will have different points of view 
about and different experiences. Please feel free to tell us about your experiences even if they 
are different from others. And we want to hear negative as well as positive comments. 

We are tape recording this session because we don’t want to miss any comments. Be assured 
that all your information is confidential and no one will be identified in the final report. I have 
consent forms here, if you want. 

The name tents are only for this discussion to help me remember your names and help you 
follow up on other peoples’ comments. If you agree or disagree, please say so. Don’t feel like 
you have to talk to me all the time, this is a discussion with you guys as the experts. 

We want to hear from each one of you, so if you haven’t said anything for awhile — I may call 
on you for your comments. If you are talking all the time, I may ask you to wait and let others 
talk. Feel free to help yourself to the food and drinks. 

OK? Let’s get started. 

First, I’d like to know who you are, your first name, what is your major, your hometown, and 
how did you end up choosing LMPub. 


APS Research Processes and Procedures 
February 2009 



Academic Pathways Study of Engineering Education; Large Midwestern Public University 

Focus Group Discussion Guide (April 5, 2007) 


Pre-discuss 

Fill out demographics (graduation year, major) 

00:00 - 
00:05 

Consent Process 

■ Hand out consent process forms and allow participants to read form 

00:05 - 
00:10 

Background Information 
■ Overview and benefits of APS study 

00:10- 

00:35 

Motivation to study engineering 

■ Why did you choose to study engineering? (motivations) 

Financial? 

Family? 

Societal benefit? 

■ Was anyone influential in your decision? Who? 

■ What previous exposure to and experience with engineering? 

00:35 - 
01:15 

Knowledge and skills of engineering 

■ Describe engineering (five words) write descriptions on post-it notes 
and group into categories on flip chart paper on wall 

■ What do you think engineers do at work ("a day in the life” “real 
world”)? 

■ What knowledge do you think engineers use in daily practice? 

- What knowledge is important for practice? 

- What do engineers need to know to practice? 

■ In what engineering-related knowledge and skills are you most 
confident yourself? 

- How confident are you in solving problems with multiple 
solutions? 

- How confident are you is your ability to apply math and science 
to solving real world problems? Experience with projects? 

01:15- 

01:30 

Institutional experience (discuss satisfaction with facilities and faculty) 

■ Overall satisfaction with LMPub 

■ Satisfaction with [name of department] (facilities) 

■ Satisfaction with your interactions in classes 

■ Satisfaction with your interactions with faculty 

■ Satisfaction with your interactions with TAs 

■ Participation in non-engineering activities 

01:30- 

01:50 

Managing workload 

■ How do you feel about your course load? (probe for pressure, stress, 
motivation to continue) 

■ What is the value of your course content? (probe for relevance) 
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■ Discuss engagement/disengagement with coursework 

■ Discuss participation in and value of extra-curricular activity 

■ How do you balance social and academic demands? 

01:50-2:00 

Anything else I should know? Concluding comments? Thank you. 
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Appendix 5-A 

Sample APPLES Response Report (day 2) 


Orchard University Response Report - SAMPLE 

Tuesday, February 12, 2008 


Strata 

Target 

11 Feb 
2:55pm 

12 Feb 
2:45 pm 

13 Feb 

14 Feb 

15 Feb 

Status 

All 

140 

106 

176 




FULFILLED 

Freshmen 

25 

13 

24 





Sophomore 

25 

21 

24 





Juniors 

25 

35 

58 




FULFILLED 

Seniors 

25 

29 

44 




FULFILLED 

Transfer students 

10 

9 

9 





Non-persisters 

25 

7 

9 





Male students 

70 

79 

100 




FULFILLED 

Female students 

25 

28 

50 




FULFILLED 

Ethnic minority students 

25 

8 

16 





International students 

25 

20 

26 




FULFILLED 

Part-time students 

10 

0 

1 






NOTES: 

• There were a relatively large number of graduate students who took the survey, please 
check recruitment methods if reminders are sent to the same lists. 

• Non-persisters and ethnic minority student responses are lower than expected. You 
may want to consider implementing Plan B for these strata. 
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Appendix 5-B 
APS Research Team 

Cumulative; compiled December 2008 


University of Washington 

Robin Adams, PhD 

Assistant Director, Research, Center for 
Engineering Learning and Teaching 
(Continued APS involvement from Purdue 
University beginning August 2005) 

Daniel Amos, PhD 
Research Scientist 

Cindy Atman, PhD 
Professor, Industrial Engineering 
CALL PI 

Sylvia Bach 

Administrator, Center for Engineering 
Learning and Teaching 

Shelley Balanko, PhD 
Program Evaluator 

Theresa Barker 

Graduate Research Assistant, Industrial 
Engineering 

Philip Bell, PhD 

Associate Professor, Educational 
Psychology 

Jim Borgford-Parnell, PhD 
Assistant Director, Research, Center for 
Engineering Learning and Teaching 

Laurie Collins, PhD 
Program Evaluator 

Lari Garrison 

Graduate Research Assistant, Education 

Partricia Gomez 
Administrative Assistant 


Lorenza Ibarrientos 
Program Operations Specialist 

Andrew Jocuns 
Research Associate 

Jana Jones 

Graduate Research Assistant 

Deborah Kilgore, PhD 
Research Scientist 

Jennifer Light, PhD 
NAE Eellow 

Angela Linse, PhD 
Research Scientist 

Tina Loucks-Jaret 

Technical Communications Specialist 

Dennis Lund 
Assistant Director 

Bayta Maring, PhD 
Program Evaluator 

Andrew Morozov 
Graduate Research Assistant 

Liz Moore, PhD 
Program Evaluator 

Quan Nguyen 
Program Assistant 

Kevin O’Connor, PhD 

Postdoctoral Research Associate, 

Educational Psychology 

(Continued APS involvement from Univ. of 

Rochester beginning August 2005) 
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Natalie Quilter 

Assistant to CAEE Director (through July 
2005) 

Ed Rhone 

Graduate Research Assistant 

Portia Sabin, PhD 
Postdoctoral Research Associate 

Jason Saleem, PhD 
Research Scientist 

Tom Satwicz 

Graduate Research Assistant 
Katie Schatz 

Graduate Research Assistant 

Carmen Smith 

Graduate Research Assistant 

Reed Stevens, PhD 
Associate Professor, Educational 
Psychology 
CAEE CO-PI 

Heather Toomey Zimmerman 
Graduate Research Assistant, Education 

Jennifer Turns, PhD 
Associate Professor, Technical 
Communication 

Ken Yasuhara 

Graduate Research Assistant, Computer 
Science and Engineering 

Colorado School of Mines 

Ravel Ammerman, PhD 
Eecturer, Electrical Engineering 

Kimberley Breaux 
Research Associate 


Monica Geist 

Graduate Research Assistant (U. Northern 
Colorado) 

Tawni Hoeglund, PhD 
Research Associate 

Heidi Eoshbaugh, PhD 

Postdoctoral Research Associate, Eiberal 

Arts and International Studies 

Ronald Miller, PhD 
Professor, Chemical Engineering 

Barbara Olds, PhD 

Office of Academic Affairs and Division of 
Eiberal Arts and International Studies 

Ruth Streveler, PhD 

Director, Center for Engineering Education 
CAEE CO-PI 

(Continued APS involvement from Purdue 
University after August 2006) 

Candace Sulzbach, PhD 
Eecturer, Civil Engineering 

Howard University 

Caryn Bailey, PhD 
Research Scientist 

Wade Boykin, PhD 
Professor, Psychology 

Karen Bland, PhD 
Research Associate 

Angela Cole, PhD 

Assistant Professor, Psychology 
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Kimarie Engerman, PhD 
Research Associate, Psychology 
(Continued APS involvement from 
University of the Virgin Islands after August 
2006) 

Lorraine Fleming, PhD 
Professor, Civil Engineering 
CAEE CO-PI 

Ashley Griffin 

Graduate Research Assistant, Psychology 
Marcus Jones 

Graduate Research Assistant 

Sislena Ledbetter 
Research Associate 

Janice McCain 
Research Associate 

David Mitchell 
Graduate Research Assistant 

Rashika Rentie 
Graduate Research Assistant 

Andrene Taylor 
Graduate Research Assistant 

Dawn Williams, PhD 
Assistant Professor, Educational 
Administration and Policy 

Stanford University 

Jeff Aldrich 
IT Consultant 

Tori Bailey 

Graduate Research Assistant, Mechanical 
Engineering 


Helen Chen, PhD 

Research Associate, Center for Innovations 
in Learning 

Mia Clark 
Technical Writer 

Laura Crenwelge 
Graduate Research Assistant 

Krista Donaldson, PhD 
Research Scientist 

Ozgiir Eris, PhD 

Research Associate, Center for Design 
Research, Mechanical Engineering 
(Continued APS involvement from Franklin 
W. Olin College beginning August 2005) 

John Feland 

Graduate Research Assistant 

Kristyn Jackson 
Graduate Research Assistant 

Larry Leifer, PhD 

Professor, Mechanical Engineering Design, 
School of Engineering 
CAEE CO-PI 

Gary Lichtenstein, EdD 
Consultant 

Sheri Sheppard, PhD 

Professor, Mechanical Engineering Design, 
School of Engineering 
CAEE CO-PI 

George Toye, PhD 

Stanford Center for Design Research 
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University of Minnesota 

Karl Smith, PhD 

Professor, Civil Engineering, Institute of 

Technology 

CO-PI, Campus PI 

(Continued APS involvement from Purdue 
University (split appointment) after June 2006) 


University of Rochester 

Lisa Perhamus 

Graduate Research Assistant 

Derek Seward 

Graduate Research Assistant 


Russ Korte 

Graduate Research Assistant 
(Continued APS involvement from University 
Texas at Tyler and University of Illinois after 
August 2007) 

Purdue University 

Holly Matusovich 
Graduate Research Assistant 


Debbie Chachra, PhD 
Research Associate, Center for Design 
Research, Mechanical Engineering 
Eranklin W. Olin College 

University of Wisconsin 

Susan Millar, PhD 
Senior Research Advisor 


Aids a Santiago 
Graduate Research Assistant 
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Undergraduate Research Assistants 


Lakshmi Akella (UW) 

Pydi Akella (UW) 

Lisa Asari (SU) 

Elisa Bovos (CSM) 

Audrey Brown (SU) 

Kaitlyn Chen (UW) 

Brittany Claar (CSM) 

Tyler Cummings-Bond (UW) 
Jeremy Donato (UW) 

Collin Donohoue (CSM) 

Joe Douglas (UW) 

Angela Du (UW) 

Lisel Forbes (HU) 

Jonathan Gabrio (SU) 

Beza Getahun (UW) 

Sandria Gray (HU) 

Johanna Hayenga (UW) 

John Haynes (CSM) 

Rachel Hsu (SU) 

Laura Julich (UW) 

Marvin Kendall (HU) 

Angela King (UW) 

Marvin Kendall (HU) 

Bradley Knapp (CSM) 

Alex Kwan (SU) 

Shante Mason (HU) 

Rukaya Mehter (UW) 

Emily Milian (CSM) 

Aprill Nelson (CSM) 

Debbie Park (SU) 

Jennifer Rees (UW) 

Charlene Reyes (UW) 

Claire Rosenbaum (SU) 
Brook Sattler (UW) 

Chloe Valencia (UW) 

Esenia Valente (CSM) 

Lamar Warren (HU) 

Jesse Whitney-Blane (CSM) 
Atrice Williams (HU) 

Mary Williams (UW) 
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